task e2e-llm-inference-service has failed: "step-fail-if-needed" exited with code 1: Error [get-kubeconfig] Found kubeconfig secret: cluster-vxb8w-admin-kubeconfig [get-kubeconfig] Wrote kubeconfig to /credentials/cluster-vxb8w-kubeconfig [get-kubeconfig] Found admin password secret: cluster-vxb8w-admin-password [get-kubeconfig] Retrieved username [get-kubeconfig] Wrote password to /credentials/cluster-vxb8w-password [get-kubeconfig] API Server URL: https://ae9386c5fe22f415e8fda6b4f93b96ec-45f0f6f9761ab0b9.elb.us-east-1.amazonaws.com:6443 [get-kubeconfig] Console URL: https://console-openshift-console.apps.29cf724e-1e5c-43f8-ba9b-d885d4b2d0d6.prod.konfluxeaas.com [clone-repo] dependabot/go_modules/go.opentelemetry.io/otel-1.41.0 [clone-repo] https://github.com/opendatahub-io/kserve [clone-repo] Cloning into '/workspace/source'... [e2e-llm-inference-service] + bash [e2e-llm-inference-service] + STATUS_FILE=/test-status/deploy-and-e2e-status [e2e-llm-inference-service] + echo failed [e2e-llm-inference-service] + COMPONENT_NAME=kserve-agent-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-agent-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:e6428204baa077dfe80627f4a0f194329877d007e326693ea30a582520538a96 [e2e-llm-inference-service] + KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:e6428204baa077dfe80627f4a0f194329877d007e326693ea30a582520538a96 [e2e-llm-inference-service] + COMPONENT_NAME=kserve-controller-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-controller-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:f83b361c5a5c631f254bc7f565660fb0e921f275f5f4455a67246ac606d20054 [e2e-llm-inference-service] + KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:f83b361c5a5c631f254bc7f565660fb0e921f275f5f4455a67246ac606d20054 [e2e-llm-inference-service] + COMPONENT_NAME=kserve-router-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-router-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:f2021b9f79dbd151723ae4c386dd28bedd0fff23f6daa0825bdc58598fc71887 [e2e-llm-inference-service] + KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:f2021b9f79dbd151723ae4c386dd28bedd0fff23f6daa0825bdc58598fc71887 [e2e-llm-inference-service] + COMPONENT_NAME=kserve-storage-initializer-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-storage-initializer-ci '.[$component_name].image' [e2e-llm-inference-service] + export STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:85f85c16414082de64de782f602228652c10c5bace08b518e803a7e911c62e5e [e2e-llm-inference-service] + STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:85f85c16414082de64de782f602228652c10c5bace08b518e803a7e911c62e5e [e2e-llm-inference-service] + COMPONENT_NAME=odh-kserve-llmisvc-controller-ci [e2e-llm-inference-service] ++ jq -r --arg component_name odh-kserve-llmisvc-controller-ci '.[$component_name].image' [e2e-llm-inference-service] + export LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:0631bd41fa4c04d3031b174c8653a10b157daeb2399bd6d846e0c994fc441e93 [e2e-llm-inference-service] + LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:0631bd41fa4c04d3031b174c8653a10b157daeb2399bd6d846e0c994fc441e93 [e2e-llm-inference-service] + ./test/scripts/openshift-ci/run-e2e-tests.sh 'llminferenceservice and cluster_cpu' 2 llm-d [e2e-llm-inference-service] Installing on cluster [e2e-llm-inference-service] Using namespace: kserve for KServe components [e2e-llm-inference-service] SKLEARN_IMAGE=quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1447 [e2e-llm-inference-service] KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:f83b361c5a5c631f254bc7f565660fb0e921f275f5f4455a67246ac606d20054 [e2e-llm-inference-service] LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:0631bd41fa4c04d3031b174c8653a10b157daeb2399bd6d846e0c994fc441e93 [e2e-llm-inference-service] KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:e6428204baa077dfe80627f4a0f194329877d007e326693ea30a582520538a96 [e2e-llm-inference-service] KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:f2021b9f79dbd151723ae4c386dd28bedd0fff23f6daa0825bdc58598fc71887 [e2e-llm-inference-service] STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:85f85c16414082de64de782f602228652c10c5bace08b518e803a7e911c62e5e [e2e-llm-inference-service] ERROR_404_ISVC_IMAGE=quay.io/opendatahub/error-404-isvc:odh-pr-1447 [e2e-llm-inference-service] SUCCESS_200_ISVC_IMAGE=quay.io/opendatahub/success-200-isvc:odh-pr-1447 [e2e-llm-inference-service] [INFO] Installing Kustomize v5.8.1 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed Kustomize v5.8.1 to /workspace/source/bin/kustomize [e2e-llm-inference-service] v5.8.1 [e2e-llm-inference-service] make: Entering directory '/workspace/source' [e2e-llm-inference-service] [INFO] Installing yq v4.52.1 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed yq v4.52.1 to /workspace/source/bin/yq [e2e-llm-inference-service] yq (https://github.com/mikefarah/yq/) version v4.52.1 [e2e-llm-inference-service] make: Leaving directory '/workspace/source' [e2e-llm-inference-service] Installing KServe Python SDK ... [e2e-llm-inference-service] [INFO] Installing uv 0.7.8 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed uv 0.7.8 to /workspace/source/bin/uv [e2e-llm-inference-service] warning: Failed to read project metadata (No `pyproject.toml` found in current directory or any parent directory). Running `uv self version` for compatibility. This fallback will be removed in the future; pass `--preview` to force an error. [e2e-llm-inference-service] uv 0.7.8 [e2e-llm-inference-service] Creating virtual environment... [e2e-llm-inference-service] warning: virtualenv's `--clear` has no effect (uv always clears the virtual environment) [e2e-llm-inference-service] Using CPython 3.9.25 interpreter at: /usr/bin/python3 [e2e-llm-inference-service] Creating virtual environment at: .venv [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] Using CPython 3.11.13 interpreter at: /usr/bin/python3.11 [e2e-llm-inference-service] Creating virtual environment at: .venv [e2e-llm-inference-service] Resolved 259 packages in 1ms [e2e-llm-inference-service] Building kserve @ file:///workspace/source/python/kserve [e2e-llm-inference-service] Downloading kubernetes (1.9MiB) [e2e-llm-inference-service] Downloading pandas (12.5MiB) [e2e-llm-inference-service] Downloading aiohttp (1.7MiB) [e2e-llm-inference-service] Downloading uvloop (3.8MiB) [e2e-llm-inference-service] Downloading pydantic-core (2.0MiB) [e2e-llm-inference-service] Downloading portforward (3.9MiB) [e2e-llm-inference-service] Downloading grpcio (6.4MiB) [e2e-llm-inference-service] Downloading grpcio-tools (2.5MiB) [e2e-llm-inference-service] Downloading mypy (17.2MiB) [e2e-llm-inference-service] Downloading pyarrow (40.1MiB) [e2e-llm-inference-service] Downloading numpy (15.7MiB) [e2e-llm-inference-service] Downloading cryptography (4.3MiB) [e2e-llm-inference-service] Downloading black (1.6MiB) [e2e-llm-inference-service] Downloading setuptools (1.2MiB) [e2e-llm-inference-service] Downloading botocore (12.9MiB) [e2e-llm-inference-service] Building timeout-sampler==1.0.3 [e2e-llm-inference-service] Building python-simple-logger==2.0.19 [e2e-llm-inference-service] Building pyasn==1.6.2 [e2e-llm-inference-service] Downloading aiohttp [e2e-llm-inference-service] Downloading black [e2e-llm-inference-service] Downloading pydantic-core [e2e-llm-inference-service] Downloading grpcio-tools [e2e-llm-inference-service] Downloading setuptools [e2e-llm-inference-service] Downloading portforward [e2e-llm-inference-service] Built python-simple-logger==2.0.19 [e2e-llm-inference-service] Downloading uvloop [e2e-llm-inference-service] Downloading cryptography [e2e-llm-inference-service] Downloading grpcio [e2e-llm-inference-service] Downloading kubernetes [e2e-llm-inference-service] Built timeout-sampler==1.0.3 [e2e-llm-inference-service] Built kserve @ file:///workspace/source/python/kserve [e2e-llm-inference-service] Downloading numpy [e2e-llm-inference-service] Downloading pandas [e2e-llm-inference-service] Built pyasn==1.6.2 [e2e-llm-inference-service] Downloading botocore [e2e-llm-inference-service] Downloading mypy [e2e-llm-inference-service] Downloading pyarrow [e2e-llm-inference-service] Prepared 100 packages in 1.85s [e2e-llm-inference-service] warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance. [e2e-llm-inference-service] If the cache and target directories are on different filesystems, hardlinking may not be supported. [e2e-llm-inference-service] If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning. [e2e-llm-inference-service] Installed 100 packages in 250ms [e2e-llm-inference-service] + aiohappyeyeballs==2.6.1 [e2e-llm-inference-service] + aiohttp==3.13.3 [e2e-llm-inference-service] + aiosignal==1.4.0 [e2e-llm-inference-service] + annotated-doc==0.0.4 [e2e-llm-inference-service] + annotated-types==0.7.0 [e2e-llm-inference-service] + anyio==4.9.0 [e2e-llm-inference-service] + attrs==25.3.0 [e2e-llm-inference-service] + avro==1.12.0 [e2e-llm-inference-service] + black==24.3.0 [e2e-llm-inference-service] + boto3==1.37.35 [e2e-llm-inference-service] + botocore==1.37.35 [e2e-llm-inference-service] + cachetools==5.5.2 [e2e-llm-inference-service] + certifi==2025.1.31 [e2e-llm-inference-service] + cffi==2.0.0 [e2e-llm-inference-service] + charset-normalizer==3.4.1 [e2e-llm-inference-service] + click==8.1.8 [e2e-llm-inference-service] + cloudevents==1.11.0 [e2e-llm-inference-service] + colorama==0.4.6 [e2e-llm-inference-service] + colorlog==6.10.1 [e2e-llm-inference-service] + coverage==7.8.0 [e2e-llm-inference-service] + cryptography==46.0.5 [e2e-llm-inference-service] + deprecation==2.1.0 [e2e-llm-inference-service] + durationpy==0.9 [e2e-llm-inference-service] + execnet==2.1.1 [e2e-llm-inference-service] + fastapi==0.121.3 [e2e-llm-inference-service] + frozenlist==1.5.0 [e2e-llm-inference-service] + google-auth==2.39.0 [e2e-llm-inference-service] + grpc-interceptor==0.15.4 [e2e-llm-inference-service] + grpcio==1.78.1 [e2e-llm-inference-service] + grpcio-testing==1.78.1 [e2e-llm-inference-service] + grpcio-tools==1.78.1 [e2e-llm-inference-service] + h11==0.16.0 [e2e-llm-inference-service] + httpcore==1.0.9 [e2e-llm-inference-service] + httptools==0.6.4 [e2e-llm-inference-service] + httpx==0.27.2 [e2e-llm-inference-service] + httpx-retries==0.4.5 [e2e-llm-inference-service] + idna==3.10 [e2e-llm-inference-service] + iniconfig==2.1.0 [e2e-llm-inference-service] + jinja2==3.1.6 [e2e-llm-inference-service] + jmespath==1.0.1 [e2e-llm-inference-service] + kserve==0.18.0rc1 (from file:///workspace/source/python/kserve) [e2e-llm-inference-service] + kubernetes==32.0.1 [e2e-llm-inference-service] + markupsafe==3.0.2 [e2e-llm-inference-service] + multidict==6.4.3 [e2e-llm-inference-service] + mypy==0.991 [e2e-llm-inference-service] + mypy-extensions==1.0.0 [e2e-llm-inference-service] + numpy==2.2.4 [e2e-llm-inference-service] + oauthlib==3.2.2 [e2e-llm-inference-service] + orjson==3.10.16 [e2e-llm-inference-service] + packaging==24.2 [e2e-llm-inference-service] + pandas==2.2.3 [e2e-llm-inference-service] + pathspec==0.12.1 [e2e-llm-inference-service] + platformdirs==4.3.7 [e2e-llm-inference-service] + pluggy==1.5.0 [e2e-llm-inference-service] + portforward==0.7.1 [e2e-llm-inference-service] + prometheus-client==0.21.1 [e2e-llm-inference-service] + propcache==0.3.1 [e2e-llm-inference-service] + protobuf==6.33.5 [e2e-llm-inference-service] + psutil==5.9.8 [e2e-llm-inference-service] + pyarrow==19.0.1 [e2e-llm-inference-service] + pyasn==1.6.2 [e2e-llm-inference-service] + pyasn1==0.6.1 [e2e-llm-inference-service] + pyasn1-modules==0.4.2 [e2e-llm-inference-service] + pycparser==2.22 [e2e-llm-inference-service] + pydantic==2.12.4 [e2e-llm-inference-service] + pydantic-core==2.41.5 [e2e-llm-inference-service] + pyjwt==2.12.1 [e2e-llm-inference-service] + pytest==7.4.4 [e2e-llm-inference-service] + pytest-asyncio==0.23.8 [e2e-llm-inference-service] + pytest-cov==5.0.0 [e2e-llm-inference-service] + pytest-httpx==0.30.0 [e2e-llm-inference-service] + pytest-xdist==3.6.1 [e2e-llm-inference-service] + python-dateutil==2.9.0.post0 [e2e-llm-inference-service] + python-dotenv==1.1.0 [e2e-llm-inference-service] + python-multipart==0.0.22 [e2e-llm-inference-service] + python-simple-logger==2.0.19 [e2e-llm-inference-service] + pytz==2025.2 [e2e-llm-inference-service] + pyyaml==6.0.2 [e2e-llm-inference-service] + requests==2.32.3 [e2e-llm-inference-service] + requests-oauthlib==2.0.0 [e2e-llm-inference-service] + rsa==4.9.1 [e2e-llm-inference-service] + s3transfer==0.11.4 [e2e-llm-inference-service] + setuptools==78.1.0 [e2e-llm-inference-service] + six==1.17.0 [e2e-llm-inference-service] + sniffio==1.3.1 [e2e-llm-inference-service] + starlette==0.49.1 [e2e-llm-inference-service] + tabulate==0.9.0 [e2e-llm-inference-service] + timeout-sampler==1.0.3 [e2e-llm-inference-service] + timing-asgi==0.3.1 [e2e-llm-inference-service] + tomlkit==0.13.2 [e2e-llm-inference-service] + typing-extensions==4.15.0 [e2e-llm-inference-service] + typing-inspection==0.4.2 [e2e-llm-inference-service] + tzdata==2025.2 [e2e-llm-inference-service] + urllib3==2.6.2 [e2e-llm-inference-service] + uvicorn==0.34.1 [e2e-llm-inference-service] + uvloop==0.21.0 [e2e-llm-inference-service] + watchfiles==1.0.5 [e2e-llm-inference-service] + websocket-client==1.8.0 [e2e-llm-inference-service] + websockets==15.0.1 [e2e-llm-inference-service] + yarl==1.20.0 [e2e-llm-inference-service] Audited 1 package in 57ms [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] Creating namespace openshift-keda... [e2e-llm-inference-service] namespace/openshift-keda created [e2e-llm-inference-service] Namespace openshift-keda created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Creating OperatorGroup openshift-keda... [e2e-llm-inference-service] operatorgroup.operators.coreos.com/openshift-keda created [e2e-llm-inference-service] OperatorGroup openshift-keda created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Creating Subscription for openshift-custom-metrics-autoscaler-operator... [e2e-llm-inference-service] subscription.operators.coreos.com/openshift-custom-metrics-autoscaler-operator created [e2e-llm-inference-service] Subscription openshift-custom-metrics-autoscaler-operator created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Waiting for openshift-custom-metrics-autoscaler-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (0/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (5/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (10/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (15/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (20/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (25/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.18.1-2 found, but not yet Succeeded (Phase: Installing). Waiting... (30/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.18.1-2 found, but not yet Succeeded (Phase: Installing). Waiting... (35/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.18.1-2 found, but not yet Succeeded (Phase: Installing). Waiting... (40/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.18.1-2 found, but not yet Succeeded (Phase: Installing). Waiting... (45/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.18.1-2 found, but not yet Succeeded (Phase: Installing). Waiting... (50/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.18.1-2 is ready (Phase: Succeeded). [e2e-llm-inference-service] --- [e2e-llm-inference-service] Applying KedaController custom resource... [e2e-llm-inference-service] Warning: resource kedacontrollers/keda is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically. [e2e-llm-inference-service] kedacontroller.keda.sh/keda configured [e2e-llm-inference-service] KedaController custom resource applied. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Allowing time for KEDA components to be provisioned by the operator ... [e2e-llm-inference-service] Waiting for KEDA Operator pod (selector: "app=keda-operator") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-operator" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-operator" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-operator" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-operator-ffbb595cb-c6xjz 1/1 Running 0 43s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-operator" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-operator-ffbb595cb-c6xjz condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-operator" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Operator pod is ready. [e2e-llm-inference-service] Waiting for KEDA Metrics API Server pod (selector: "app=keda-metrics-apiserver") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-metrics-apiserver" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-metrics-apiserver" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-metrics-apiserver" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-metrics-apiserver-7c9f485588-92n7z 1/1 Running 0 48s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-metrics-apiserver" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-metrics-apiserver-7c9f485588-92n7z condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-metrics-apiserver" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Metrics API Server pod is ready. [e2e-llm-inference-service] Waiting for KEDA Webhook pod (selector: "app=keda-admission-webhooks") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-admission-webhooks" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-admission-webhooks" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-admission-webhooks" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-admission-cf49989db-ldftr 1/1 Running 0 53s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-admission-webhooks" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-admission-cf49989db-ldftr condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-admission-webhooks" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Webhook pod is ready. [e2e-llm-inference-service] --- [e2e-llm-inference-service] ✅ KEDA deployment script finished successfully. [e2e-llm-inference-service] 🔧 Configuration: [e2e-llm-inference-service] KServe deployment: ❌ disabled [e2e-llm-inference-service] Kuadrant deployment: ✅ enabled [e2e-llm-inference-service] [e2e-llm-inference-service] Checking OpenShift server version...(4.20.19) [e2e-llm-inference-service] 🎯 Server version (4.20.19) is 4.19.9 or higher - continue with the script [e2e-llm-inference-service] ⏳ Installing cert-manager [e2e-llm-inference-service] namespace/cert-manager-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/cert-manager-operator created [e2e-llm-inference-service] subscription.operators.coreos.com/openshift-cert-manager-operator created [e2e-llm-inference-service] Waiting for openshift-cert-manager-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (0/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (5/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (10/300) [e2e-llm-inference-service] CSV cert-manager-operator.v1.19.0 found, but not yet Succeeded (Phase: Installing). Waiting... (15/300) [e2e-llm-inference-service] CSV cert-manager-operator.v1.19.0 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD certificates.cert-manager.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD certificates.cert-manager.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io condition met [e2e-llm-inference-service] ✅ Cert-manager installed [e2e-llm-inference-service] ⏳ Installing openshift-lws-operator [e2e-llm-inference-service] namespace/openshift-lws-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/leader-worker-set created [e2e-llm-inference-service] subscription.operators.coreos.com/leader-worker-set created [e2e-llm-inference-service] Waiting for leader-worker-set CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (0/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (5/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (10/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (15/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (20/300) [e2e-llm-inference-service] CSV leader-worker-set.v1.0.0 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD leaderworkersetoperators.operator.openshift.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD leaderworkersetoperators.operator.openshift.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/leaderworkersetoperators.operator.openshift.io condition met [e2e-llm-inference-service] leaderworkersetoperator.operator.openshift.io/cluster created [e2e-llm-inference-service] ⏳ waiting for openshift-lws-operator to be ready.… [e2e-llm-inference-service] Waiting for pod -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" to be created... [e2e-llm-inference-service] Pod -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" found. [e2e-llm-inference-service] Current pods for -l "name=openshift-lws-operator" in namespace "openshift-lws-operator": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] openshift-lws-operator-bfc7f696d-w7ggm 1/1 Running 0 12s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" to become ready... [e2e-llm-inference-service] pod/openshift-lws-operator-bfc7f696d-w7ggm condition met [e2e-llm-inference-service] Pod(s) -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" are ready. [e2e-llm-inference-service] ✅ openshift-lws-operator installed [e2e-llm-inference-service] gatewayclass.gateway.networking.k8s.io/openshift-default created [e2e-llm-inference-service] Waiting for pod -l "app=istiod" in namespace "openshift-ingress" to be created... [e2e-llm-inference-service] Pod -l "app=istiod" in namespace "openshift-ingress" found. [e2e-llm-inference-service] Current pods for -l "app=istiod" in namespace "openshift-ingress": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] istiod-openshift-gateway-7cd77c7ffd-7z694 1/1 Running 0 5s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "app=istiod" in namespace "openshift-ingress" to become ready... [e2e-llm-inference-service] pod/istiod-openshift-gateway-7cd77c7ffd-7z694 condition met [e2e-llm-inference-service] Pod(s) -l "app=istiod" in namespace "openshift-ingress" are ready. [e2e-llm-inference-service] ⏳ Creating a Gateway [e2e-llm-inference-service] Error from server (AlreadyExists): namespaces "openshift-ingress" already exists [e2e-llm-inference-service] gateway.gateway.networking.k8s.io/openshift-ai-inference created [e2e-llm-inference-service] Waiting for pod -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" to be created... [e2e-llm-inference-service] Pod -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" found. [e2e-llm-inference-service] Current pods for -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] openshift-ai-inference-openshift-default-7c5447bb76-q9nvg 1/1 Running 0 6s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" to become ready... [e2e-llm-inference-service] pod/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg condition met [e2e-llm-inference-service] Pod(s) -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" are ready. [e2e-llm-inference-service] ⏳ Installing RHCL(Kuadrant) operator [e2e-llm-inference-service] namespace/kuadrant-system created [e2e-llm-inference-service] subscription.operators.coreos.com/rhcl-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/kuadrant created [e2e-llm-inference-service] Waiting for rhcl-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (0/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (5/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (10/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (15/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.3.2 found, but not yet Succeeded (Phase: Pending). Waiting... (20/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.3.2 found, but not yet Succeeded (Phase: Installing). Waiting... (25/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.3.2 found, but not yet Succeeded (Phase: Installing). Waiting... (30/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.3.2 found, but not yet Succeeded (Phase: Installing). Waiting... (35/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.3.2 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD kuadrants.kuadrant.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD kuadrants.kuadrant.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/kuadrants.kuadrant.io condition met [e2e-llm-inference-service] Waiting for apiserver discovery /apis/kuadrant.io/v1beta1 to list kuadrants (timeout: 120s)… [e2e-llm-inference-service] Discovery for kuadrant.io/v1beta1 includes kuadrants. [e2e-llm-inference-service] ⏳ sleeping 30s after discovery (RESTMapper can trail discovery)… [e2e-llm-inference-service] kuadrant.kuadrant.io/kuadrant created [e2e-llm-inference-service] ⏳ waiting for Kuadrant Ready (attempt 1/2, timeout 5m)… [e2e-llm-inference-service] kuadrant.kuadrant.io/kuadrant condition met [e2e-llm-inference-service] Waiting for pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" to be created... [e2e-llm-inference-service] Pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" found. [e2e-llm-inference-service] Current pods for -l "control-plane=authorino-operator" in namespace "kuadrant-system": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] authorino-operator-7587b89b76-dnrb2 1/1 Running 0 81s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" to become ready... [e2e-llm-inference-service] pod/authorino-operator-7587b89b76-dnrb2 condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" are ready. [e2e-llm-inference-service] ⏳ waiting for authorino service to be created... [e2e-llm-inference-service] service/authorino-authorino-authorization condition met [e2e-llm-inference-service] service/authorino-authorino-authorization annotated [e2e-llm-inference-service] Warning: resource authorinos/authorino is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically. [e2e-llm-inference-service] authorino.operator.authorino.kuadrant.io/authorino configured [e2e-llm-inference-service] Waiting for pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" to be created... [e2e-llm-inference-service] Pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" found. [e2e-llm-inference-service] Current pods for -l "control-plane=authorino-operator" in namespace "kuadrant-system": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] authorino-operator-7587b89b76-dnrb2 1/1 Running 0 89s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" to become ready... [e2e-llm-inference-service] pod/authorino-operator-7587b89b76-dnrb2 condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" are ready. [e2e-llm-inference-service] ✅ kuadrant(authorino) installed [e2e-llm-inference-service] Now using project "kserve" on server "https://ae9386c5fe22f415e8fda6b4f93b96ec-45f0f6f9761ab0b9.elb.us-east-1.amazonaws.com:6443". [e2e-llm-inference-service] [e2e-llm-inference-service] You can add applications to this project with the 'new-app' command. For example, try: [e2e-llm-inference-service] [e2e-llm-inference-service] oc new-app rails-postgresql-example [e2e-llm-inference-service] [e2e-llm-inference-service] to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application: [e2e-llm-inference-service] [e2e-llm-inference-service] kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname [e2e-llm-inference-service] [e2e-llm-inference-service] ⏳ Installing KServe with SeaweedFS [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencemodelrewrites.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceobjectives.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepoolimports.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied [e2e-llm-inference-service] ⏳ Waiting for CRDs to be established [e2e-llm-inference-service] Waiting for CRD inferenceservices.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD inferenceservices.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD llminferenceserviceconfigs.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD llminferenceserviceconfigs.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD clusterstoragecontainers.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD clusterstoragecontainers.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD datascienceclusters.datasciencecluster.opendatahub.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD datascienceclusters.datasciencecluster.opendatahub.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io condition met [e2e-llm-inference-service] ⏳ Applying all resources... [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencemodelrewrites.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceobjectives.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepoolimports.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied [e2e-llm-inference-service] serviceaccount/kserve-controller-manager serverside-applied [e2e-llm-inference-service] serviceaccount/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-leader-election-role serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/llmisvc-leader-election-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-admin serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-edit serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-metrics-reader-cluster-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-proxy-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-view serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/llmisvc-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-llmisvc-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-manager-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-proxy-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/llmisvc-manager-rolebinding serverside-applied [e2e-llm-inference-service] configmap/inferenceservice-config serverside-applied [e2e-llm-inference-service] configmap/kserve-parameters serverside-applied [e2e-llm-inference-service] secret/kserve-webhook-server-secret serverside-applied [e2e-llm-inference-service] secret/mlpipeline-s3-artifact serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-metrics-service serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-service serverside-applied [e2e-llm-inference-service] service/kserve-webhook-server-service serverside-applied [e2e-llm-inference-service] service/llmisvc-controller-manager-service serverside-applied [e2e-llm-inference-service] service/llmisvc-webhook-server-service serverside-applied [e2e-llm-inference-service] service/s3-service serverside-applied [e2e-llm-inference-service] deployment.apps/kserve-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/seaweedfs serverside-applied [e2e-llm-inference-service] networkpolicy.networking.k8s.io/kserve-controller-manager serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] clusterstoragecontainer.serving.kserve.io/default serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-router-route serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-scheduler serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-worker-data-parallel serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferencegraph.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceserviceconfig.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/servingruntime.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/trainedmodel.serving.kserve.io serverside-applied [e2e-llm-inference-service] ⏳ Waiting for llmisvc-controller-manager to be ready... [e2e-llm-inference-service] Waiting for pod -l "control-plane=llmisvc-controller-manager" in namespace "kserve" to be created... [e2e-llm-inference-service] Pod -l "control-plane=llmisvc-controller-manager" in namespace "kserve" found. [e2e-llm-inference-service] Current pods for -l "control-plane=llmisvc-controller-manager" in namespace "kserve": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] llmisvc-controller-manager-6c9547c57-hr9ff 0/1 ContainerCreating 0 5s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=llmisvc-controller-manager" in namespace "kserve" to become ready... [e2e-llm-inference-service] pod/llmisvc-controller-manager-6c9547c57-hr9ff condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=llmisvc-controller-manager" in namespace "kserve" are ready. [e2e-llm-inference-service] ⏳ Re-applying LLMInferenceServiceConfig resources with webhook validation... [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-decode-template is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-template serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-decode-worker-data-parallel is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-worker-data-parallel serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-prefill-template is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-template serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-prefill-worker-data-parallel is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-router-route serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-scheduler is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-scheduler serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-template is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-worker-data-parallel is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-worker-data-parallel serverside-applied [e2e-llm-inference-service] Installing DSC/DSCI resources... [e2e-llm-inference-service] dscinitialization.dscinitialization.opendatahub.io/test-dsci created [e2e-llm-inference-service] datasciencecluster.datasciencecluster.opendatahub.io/test-dsc created [e2e-llm-inference-service] Patching ingress domain, markers: llminferenceservice and cluster_cpu [e2e-llm-inference-service] configmap/inferenceservice-config patched [e2e-llm-inference-service] pod "kserve-controller-manager-74fc8f6f96-k8bvx" deleted [e2e-llm-inference-service] datasciencecluster.datasciencecluster.opendatahub.io/test-dsc patched [e2e-llm-inference-service] waiting kserve-controller get ready... [e2e-llm-inference-service] pod/kserve-controller-manager-74fc8f6f96-bfwqh condition met [e2e-llm-inference-service] Installing ODH Model Controller manually with PR images [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/accounts.nim.opendatahub.io created [e2e-llm-inference-service] serviceaccount/model-serving-api created [e2e-llm-inference-service] serviceaccount/odh-model-controller created [e2e-llm-inference-service] role.rbac.authorization.k8s.io/leader-election-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-prometheus-k8s created [e2e-llm-inference-service] Warning: resource clusterroles/metrics-reader is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically. [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/metrics-reader configured [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/model-serving-api created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/odh-model-controller-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/proxy-role created [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/leader-election-rolebinding created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/model-serving-api created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/odh-model-controller-rolebinding-opendatahub created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/proxy-rolebinding created [e2e-llm-inference-service] configmap/odh-model-controller-parameters created [e2e-llm-inference-service] service/model-serving-api created [e2e-llm-inference-service] service/odh-model-controller-metrics-service created [e2e-llm-inference-service] service/odh-model-controller-webhook-service created [e2e-llm-inference-service] deployment.apps/model-serving-api created [e2e-llm-inference-service] deployment.apps/odh-model-controller created [e2e-llm-inference-service] servicemonitor.monitoring.coreos.com/model-serving-api-metrics created [e2e-llm-inference-service] servicemonitor.monitoring.coreos.com/odh-model-controller-metrics-monitor created [e2e-llm-inference-service] template.template.openshift.io/guardrails-detector-huggingface-serving-template created [e2e-llm-inference-service] template.template.openshift.io/kserve-ovms created [e2e-llm-inference-service] template.template.openshift.io/mlserver-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-x86-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cuda-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-gaudi-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-multinode-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-rocm-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-ppc64le-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-s390x-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-x86-runtime-template created [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/mutating.odh-model-controller.opendatahub.io created [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/validating.odh-model-controller.opendatahub.io created [e2e-llm-inference-service] Waiting for deployment "odh-model-controller" rollout to finish: 0 of 1 updated replicas are available... [e2e-llm-inference-service] deployment "odh-model-controller" successfully rolled out [e2e-llm-inference-service] Add testing models to SeaweedFS S3 storage ... [e2e-llm-inference-service] Waiting for SeaweedFS deployment to be ready... [e2e-llm-inference-service] deployment "seaweedfs" successfully rolled out [e2e-llm-inference-service] S3 init job not completed, re-creating... [e2e-llm-inference-service] job.batch/s3-init created [e2e-llm-inference-service] Waiting for S3 init job to complete... [e2e-llm-inference-service] job.batch/s3-init condition met [e2e-llm-inference-service] networkpolicy.networking.k8s.io/allow-all created [e2e-llm-inference-service] Prepare CI namespace and install ServingRuntimes [e2e-llm-inference-service] Setting up CI namespace: kserve-ci-e2e-test [e2e-llm-inference-service] Tearing down CI namespace: kserve-ci-e2e-test [e2e-llm-inference-service] Namespace kserve-ci-e2e-test does not exist, skipping deletion [e2e-llm-inference-service] CI namespace teardown complete [e2e-llm-inference-service] Creating namespace kserve-ci-e2e-test [e2e-llm-inference-service] namespace/kserve-ci-e2e-test created [e2e-llm-inference-service] Applying S3 artifact secret [e2e-llm-inference-service] secret/mlpipeline-s3-artifact created [e2e-llm-inference-service] Applying storage-config secret [e2e-llm-inference-service] secret/storage-config created [e2e-llm-inference-service] Creating odh-trusted-ca-bundle configmap [e2e-llm-inference-service] configmap/odh-trusted-ca-bundle created [e2e-llm-inference-service] Installing ServingRuntimes [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-huggingfaceserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-huggingfaceserver-multinode created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-lgbserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-mlserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-paddleserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-pmmlserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-predictiveserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-sklearnserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-tensorflow-serving created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-torchserve created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-tritonserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-xgbserver created [e2e-llm-inference-service] CI namespace setup complete [e2e-llm-inference-service] Setup complete [e2e-llm-inference-service] === E2E cluster / operator summary === [e2e-llm-inference-service] Client Version: 4.20.11 [e2e-llm-inference-service] Kustomize Version: v5.6.0 [e2e-llm-inference-service] Server Version: 4.20.19 [e2e-llm-inference-service] Kubernetes Version: v1.33.9 [e2e-llm-inference-service] ClusterVersion desired: 4.20.19 [e2e-llm-inference-service] ClusterVersion history (latest): 4.20.19 (Completed) [e2e-llm-inference-service] CSVs in kuadrant-system: [e2e-llm-inference-service] authorino-operator.v1.3.0 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.19.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.3.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.3.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.3.2 Succeeded [e2e-llm-inference-service] servicemeshoperator3.v3.1.0 Succeeded [e2e-llm-inference-service] CSVs in openshift-keda: [e2e-llm-inference-service] authorino-operator.v1.3.0 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.19.0 Succeeded [e2e-llm-inference-service] custom-metrics-autoscaler.v2.18.1-2 Succeeded [e2e-llm-inference-service] dns-operator.v1.3.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.3.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.3.2 Succeeded [e2e-llm-inference-service] servicemeshoperator3.v3.1.0 Succeeded [e2e-llm-inference-service] CSVs in cert-manager-operator: [e2e-llm-inference-service] authorino-operator.v1.3.0 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.19.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.3.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.3.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.3.2 Succeeded [e2e-llm-inference-service] servicemeshoperator3.v3.1.0 Succeeded [e2e-llm-inference-service] CSVs in openshift-lws-operator: [e2e-llm-inference-service] authorino-operator.v1.3.0 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.19.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.3.0 Succeeded [e2e-llm-inference-service] leader-worker-set.v1.0.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.3.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.3.2 Succeeded [e2e-llm-inference-service] servicemeshoperator3.v3.1.0 Succeeded [e2e-llm-inference-service] CSVs in openshift-operators (ODH / shared operators, filtered): [e2e-llm-inference-service] authorino-operator.v1.3.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.3.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.3.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.3.2 Succeeded [e2e-llm-inference-service] servicemeshoperator3.v3.1.0 Succeeded [e2e-llm-inference-service] Kuadrant / Authorino (diagnostics): [e2e-llm-inference-service] CRD kuadrants.kuadrant.io versions: v1beta1 served=true storage=true [e2e-llm-inference-service] Subscriptions in kuadrant-system: [e2e-llm-inference-service] authorino-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators authorino-operator.v1.3.0 [e2e-llm-inference-service] dns-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators dns-operator.v1.3.0 [e2e-llm-inference-service] limitador-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators limitador-operator.v1.3.0 [e2e-llm-inference-service] rhcl-operator stable redhat-operators rhcl-operator.v1.3.2 [e2e-llm-inference-service] Kuadrant CR conditions (kuadrant/kuadrant-system): [e2e-llm-inference-service] Ready=True (Ready) [e2e-llm-inference-service] === End E2E cluster / operator summary === [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] REQUESTS_CA_BUNDLE=-----BEGIN CERTIFICATE----- [e2e-llm-inference-service] MIIDPDCCAiSgAwIBAgIIeXZXwQ4D1uowDQYJKoZIhvcNAQELBQAwJjESMBAGA1UE [e2e-llm-inference-service] CxMJb3BlbnNoaWZ0MRAwDgYDVQQDEwdyb290LWNhMB4XDTI2MDQyNDIxMTAxN1oX [e2e-llm-inference-service] DTM2MDQyMTIxMTAxN1owJjESMBAGA1UECxMJb3BlbnNoaWZ0MRAwDgYDVQQDEwdy [e2e-llm-inference-service] b290LWNhMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAq+40Zw/oOFWj [e2e-llm-inference-service] P6TXN4HwNIoRPWcKib7tNUsmCaLkH5FDIdmwQEjsSPrbUdZuj6zl1IOFlHpSK6uQ [e2e-llm-inference-service] BsjnWdDMhjlabXe+1ATVOohqN79Ghiokr9KYV7Yj2aFgXeGPhFHhNPG3bHPlJntq [e2e-llm-inference-service] bAsmFAmdPKbjMT1W7vias9xDaqQeEjdZuDa6LK5Mu9E6Cb3ZWRqWtsQNqL+uz4R7 [e2e-llm-inference-service] fS9hLGMJAjIjyO0z/MSlL+A5nQyLqZvidi6OwP6fyUYNl7h37ZMT+Wy7f+Xmf1Ta [e2e-llm-inference-service] YcwYTnjLsT3IqRvWcOTg1LnNDTVSn26dAuRQy+J0vMjdea1u0ehRtxr8ZKq2FUAY [e2e-llm-inference-service] G1h6aAVQZQIDAQABo24wbDAOBgNVHQ8BAf8EBAMCAqQwDwYDVR0TAQH/BAUwAwEB [e2e-llm-inference-service] /zBJBgNVHQ4EQgRAUtvuagmW/4aOLIHNo4Hs1iY3QIIc8A9qwoP4NBbvEE7QOv2X [e2e-llm-inference-service] yIZZsEuaPO1xIZshPRqbUkOTtfzEQOEC92JCkzANBgkqhkiG9w0BAQsFAAOCAQEA [e2e-llm-inference-service] BPJ+8RnYWWdpDnP+uj/xmwyrkwwVeRoecNdmYiN6mXK2Dd+Xmpphyu1rZYpg8eln [e2e-llm-inference-service] k/4VNQRAgXbGYEpEuAkK5iiPPiAz4mqsyZFACYTa00rK7/xx8/tPa6IgUabQq49f [e2e-llm-inference-service] k2tHigsH2DhTpW/UsF0KmHlX8gCWtb/a+ihYmfbyR0t8rG1F0mTMz4bc+0cBEXLS [e2e-llm-inference-service] bwEyvhKRHNuorm1jyjsVUds2b/6YyS/Wycpx72g78Gul6M/IlEq9mGCyDgxsjfKV [e2e-llm-inference-service] q5eUMFAINadCyBLRLUyIbYo4UKB2MzsnM73vTPafZ4G2ZTpSghhM5ZkMps+pKjqI [e2e-llm-inference-service] Wq1b1eBlSMpaQPpANIwFbQ== [e2e-llm-inference-service] -----END CERTIFICATE----- [e2e-llm-inference-service] -----BEGIN CERTIFICATE----- [e2e-llm-inference-service] MIIEADCCAuigAwIBAgIIS9tPudmSXCAwDQYJKoZIhvcNAQELBQAwJjESMBAGA1UE [e2e-llm-inference-service] CxMJb3BlbnNoaWZ0MRAwDgYDVQQDEwdyb290LWNhMB4XDTI2MDQyNDIxMTA0N1oX [e2e-llm-inference-service] DTI3MDQyNDIxMTA0N1owMDESMBAGA1UEChMJb3BlbnNoaWZ0MRowGAYDVQQDExFv [e2e-llm-inference-service] cGVuc2hpZnQtaW5ncmVzczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB [e2e-llm-inference-service] AOhiLsTyVtiW7PRkr9uMeVEylQywhn9JGYF6srI9yMhgNkIobYk5FfUvgF4NcvXc [e2e-llm-inference-service] VKneWWNu+X1AQ33g3lBr5fMujZv1sjj4RlsgoxZaT1UhrvUbLZnPSjT+TaF1QoCa [e2e-llm-inference-service] FlxzFOr7mNpv7xaUOFrzc8FE8j7sf+b95xrWRWaKM1cHmFZYQtntx78cRcI7t12i [e2e-llm-inference-service] SNUoxuYkogfIfOFEbYsguMnK+Apgf3jJYDocd1B+4kpp4TJN5kfmIgseueWuLucX [e2e-llm-inference-service] 8A/7zyB8XGxRyJurtfrInBFhcNe5+Tz9yIgR5aR2FGqLcmkYodaUZ4zMhK64aOQJ [e2e-llm-inference-service] 7FOhRLquitMAEQLRDV2Jc7UCAwEAAaOCASYwggEiMA4GA1UdDwEB/wQEAwIFoDAd [e2e-llm-inference-service] BgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwEwDAYDVR0TAQH/BAIwADBJBgNV [e2e-llm-inference-service] HQ4EQgRAQ3xEYWTewCYB40NH53QP/scqiJYe9S8ZZdmTagy34Px8ytLoRITnvlGY [e2e-llm-inference-service] Djo7vRzU53Thu3aB7ewDJu6uFvOZxjBLBgNVHSMERDBCgEBS2+5qCZb/ho4sgc2j [e2e-llm-inference-service] gezWJjdAghzwD2rCg/g0Fu8QTtA6/ZfIhlmwS5o87XEhmyE9GptSQ5O1/MRA4QL3 [e2e-llm-inference-service] YkKTMEsGA1UdEQREMEKCQCouYXBwcy4yOWNmNzI0ZS0xZTVjLTQzZjgtYmE5Yi1k [e2e-llm-inference-service] ODg1ZDRiMmQwZDYucHJvZC5rb25mbHV4ZWFhcy5jb20wDQYJKoZIhvcNAQELBQAD [e2e-llm-inference-service] ggEBACcJKMM7A7If66Y+jCk71dIs4DIMq0rbQSUgdUyNsR4JFBUGtlF3+LulcGAC [e2e-llm-inference-service] 5ZM6YZOLv0SyYi+GRx8qb06/pDjZBPi+uqTv9izkFg2J8u7Mg4E0I6Rbuc1m1b/l [e2e-llm-inference-service] xUk3kiN+AOpTNiFS5dPFCkNtcN+s4XBYHvl9AuHZFIqeCc/RMNvwngg73KE8RiGw [e2e-llm-inference-service] otksPYJ3X3Lc5mee9lIvG4azIx7YNIH00q0+aPINFVSXmNoQH9vDoI60R2iTtVnN [e2e-llm-inference-service] pW6zgtYYbDoYQ8QxiQ1pClLUXC4YYxwr1kD/paCS+1s0cPLvV9mydCqNQhLwLZLu [e2e-llm-inference-service] AFpdjSqYH4LfLX76EWgnoyk6zy0= [e2e-llm-inference-service] -----END CERTIFICATE----- [e2e-llm-inference-service] [e2e-llm-inference-service] -----BEGIN CERTIFICATE----- [e2e-llm-inference-service] MIIDUTCCAjmgAwIBAgIIYxCK1Uo5nKswDQYJKoZIhvcNAQELBQAwNjE0MDIGA1UE [e2e-llm-inference-service] Awwrb3BlbnNoaWZ0LXNlcnZpY2Utc2VydmluZy1zaWduZXJAMTc3NzA2NTQwOTAe [e2e-llm-inference-service] Fw0yNjA0MjQyMTE2NDlaFw0yODA2MjIyMTE2NTBaMDYxNDAyBgNVBAMMK29wZW5z [e2e-llm-inference-service] aGlmdC1zZXJ2aWNlLXNlcnZpbmctc2lnbmVyQDE3NzcwNjU0MDkwggEiMA0GCSqG [e2e-llm-inference-service] SIb3DQEBAQUAA4IBDwAwggEKAoIBAQDTNN3lfW6hJ05se9RgiemqiLE6Zahke9Pk [e2e-llm-inference-service] qp8X68/+2HsW/qbLXHpo8APhYUqwq4Ou3YfHbs9OP60Dmv0hbQJzwqZ807JA9Co3 [e2e-llm-inference-service] bWCSdDD+DbPTFJoeJJodIaWdVAY6Hea+L1Au5qTp/e5VEWSyL44OudZ56Eji770S [e2e-llm-inference-service] 9/MvsHPzsIbr9S0snIv2p3dqDsII7VlpSAd8jgE3j24cqlvLKHvn9lK/H1ko+kVL [e2e-llm-inference-service] WzSZTLeJJwTQPCFwQIv1BvTLysen9PAnHMHSoV0i5OnxL4+j9Eywlm1Nt5SOvU4D [e2e-llm-inference-service] gWBScIS88Kef/GNjPn/Hdg+NS4UiMaklv5YtqKtmWky0XwyIGscRAgMBAAGjYzBh [e2e-llm-inference-service] MA4GA1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRl5cAB [e2e-llm-inference-service] UTAB/KCjtceXgaGqIJoy/DAfBgNVHSMEGDAWgBRl5cABUTAB/KCjtceXgaGqIJoy [e2e-llm-inference-service] /DANBgkqhkiG9w0BAQsFAAOCAQEADVloTBSgn6A2LQ0432kUx9piXYaRvEgPucbP [e2e-llm-inference-service] dgIYCG5ScwHG2DJN1D1hzx9XN9Q8Rt7RMGNbKPJih3WRiToI7sqi9Yihr3qFvhxS [e2e-llm-inference-service] E8gczHYequ/RcoqljcTVsVH4r6oS21p+E9aNO53fq/CUS0Hl6ZnW1FSUqxwCB2W+ [e2e-llm-inference-service] xXwjAAdiCRNx9ALiiZ+pPGSwLI6udd2C2v4K7lkQn26rgcJ3tSl+tz5HtqBCX9Jt [e2e-llm-inference-service] NHwTXq+C95t3MwarZ3NQmlEgDH4fCSm/cWhaOTBb4aCY4gigwdY5XsG+2JqS7C5J [e2e-llm-inference-service] Mjo9QmEawlKjTEAYi6nx/xqXi6XoUlqQL+VroGAJAtW5g9tKjw== [e2e-llm-inference-service] -----END CERTIFICATE----- [e2e-llm-inference-service] Run E2E tests: llminferenceservice and cluster_cpu [e2e-llm-inference-service] Starting E2E functional tests ... [e2e-llm-inference-service] Parallelism requested for pytest is 2 [e2e-llm-inference-service] ============================= test session starts ============================== [e2e-llm-inference-service] platform linux -- Python 3.11.13, pytest-7.4.4, pluggy-1.5.0 -- /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] cachedir: .pytest_cache [e2e-llm-inference-service] rootdir: /workspace/source/test/e2e [e2e-llm-inference-service] configfile: pytest.ini [e2e-llm-inference-service] plugins: asyncio-0.23.8, anyio-4.9.0, xdist-3.6.1, httpx-0.30.0, cov-5.0.0 [e2e-llm-inference-service] asyncio: mode=Mode.STRICT [e2e-llm-inference-service] created: 2/2 workers [e2e-llm-inference-service] 2 workers [22 items] [e2e-llm-inference-service] [e2e-llm-inference-service] scheduling tests via WorkStealingScheduling [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-with-section-name] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] 2026-04-24 21:26:42.477 5346 kserve INFO [conftest.py:configure_logger():40] Logger configured [e2e-llm-inference-service] 2026-04-24 21:26:42.477 5343 kserve INFO [conftest.py:configure_logger():40] Logger configured [e2e-llm-inference-service] 2026-04-24 21:26:42.490 5343 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:26:42.490 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:26:42.517 5343 kserve.trace Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-04-24 21:26:42.517 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():62] Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-04-24 21:26:42.523 5343 kserve.trace ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] 2026-04-24 21:26:42.523 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():70] ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-with-section-name] [e2e-llm-inference-service] llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-without-section-name] 2026-04-24 21:27:17.646 5343 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:27:17.646 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:27:17.695 5343 kserve.trace ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-04-24 21:27:17.695 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():57] ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-without-section-name] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] 2026-04-24 21:27:57.254 5343 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:27:57.254 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:27:57.310 5343 kserve.trace ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-04-24 21:27:57.310 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():57] ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] 2026-04-24 21:35:12.501 5343 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:35:12.501 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:35:12.552 5343 kserve.trace ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-04-24 21:35:12.552 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():57] ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-04-24 21:35:12.553 5343 kserve.trace Checking HttpRoute router-route-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:35:12.553 5343 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:35:12.561 5343 kserve.trace Resource not found, creating HttpRoute router-route-1 [e2e-llm-inference-service] 2026-04-24 21:35:12.561 5343 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-1 [e2e-llm-inference-service] 2026-04-24 21:35:12.574 5343 kserve.trace ✓ Successfully created HttpRoute router-route-1 [e2e-llm-inference-service] 2026-04-24 21:35:12.574 5343 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-1 [e2e-llm-inference-service] 2026-04-24 21:35:12.574 5343 kserve.trace Checking HttpRoute router-route-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:35:12.574 5343 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:35:12.577 5343 kserve.trace Resource not found, creating HttpRoute router-route-2 [e2e-llm-inference-service] 2026-04-24 21:35:12.577 5343 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-2 [e2e-llm-inference-service] 2026-04-24 21:35:12.588 5343 kserve.trace ✓ Successfully created HttpRoute router-route-2 [e2e-llm-inference-service] 2026-04-24 21:35:12.588 5343 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-2 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] FAILED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha1_to_v1alpha2_conversion [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha1_to_v1alpha2_conversion [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha2_to_v1alpha1_conversion [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha2_to_v1alpha1_conversion [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_criticality_preservation_via_annotations [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_criticality_preservation_via_annotations [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_lora_criticality_preservation [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_lora_criticality_preservation [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_round_trip_conversion_preserves_fields [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_round_trip_conversion_preserves_fields [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_storage_version_migration.py::TestStorageVersionMigration::test_storage_version_migration_after_simulated_upgrade [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_storage_version_migration.py::TestStorageVersionMigration::test_storage_version_migration_after_simulated_upgrade [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] FAILED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] 2026-04-24 21:55:11.518 5343 kserve.trace Checking Gateway router-gateway-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:55:11.518 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:55:11.555 5343 kserve.trace Resource not found, creating Gateway router-gateway-2 [e2e-llm-inference-service] 2026-04-24 21:55:11.555 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():62] Resource not found, creating Gateway router-gateway-2 [e2e-llm-inference-service] 2026-04-24 21:55:11.561 5343 kserve.trace ✓ Successfully created Gateway router-gateway-2 [e2e-llm-inference-service] 2026-04-24 21:55:11.561 5343 kserve.trace INFO [gw_api.py:create_or_update_gateway():70] ✓ Successfully created Gateway router-gateway-2 [e2e-llm-inference-service] 2026-04-24 21:55:11.561 5343 kserve.trace Checking HttpRoute router-route-3 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:55:11.561 5343 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-3 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:55:11.564 5343 kserve.trace Resource not found, creating HttpRoute router-route-3 [e2e-llm-inference-service] 2026-04-24 21:55:11.564 5343 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-3 [e2e-llm-inference-service] 2026-04-24 21:55:11.574 5343 kserve.trace ✓ Successfully created HttpRoute router-route-3 [e2e-llm-inference-service] 2026-04-24 21:55:11.574 5343 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-3 [e2e-llm-inference-service] 2026-04-24 21:55:11.574 5343 kserve.trace Checking HttpRoute router-route-4 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:55:11.574 5343 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-4 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-04-24 21:55:11.580 5343 kserve.trace Resource not found, creating HttpRoute router-route-4 [e2e-llm-inference-service] 2026-04-24 21:55:11.580 5343 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-4 [e2e-llm-inference-service] 2026-04-24 21:55:11.586 5343 kserve.trace ✓ Successfully created HttpRoute router-route-4 [e2e-llm-inference-service] 2026-04-24 21:55:11.586 5343 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-4 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [e2e-llm-inference-service] =================================== FAILURES =================================== [e2e-llm-inference-service] _ test_llm_inference_service[router-managed-scheduler-with-replicas-workload-llmd-simulator] _ [e2e-llm-inference-service] [gw1] linux -- Python 3.11.13 /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'scheduler-with-replicas', 'workload-llmd-simulator'], prompt='KServe is a', ser... {'name': 'workload-llmd-simulator-schedul-609398f0'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] @pytest.mark.parametrize( [e2e-llm-inference-service] "test_case", [e2e-llm-inference-service] [ [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-gateway-ref", [e2e-llm-inference-service] "router-with-managed-route", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/completions", [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=create_response_assertion(with_field="choices"), [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[0]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-custom-route-timeout", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="custom-route-timeout-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-refs", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="router-with-refs-test", [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[0]], [e2e-llm-inference-service] routes=[ROUTER_ROUTES[0], ROUTER_ROUTES[1]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=["router-managed", "workload-pd-cpu", "model-fb-opt-125m"], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-custom-route-timeout-pd", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-pd-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] service_name="custom-route-timeout-pd-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-refs-pd", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-pd-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] service_name="router-with-refs-pd-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[1]], [e2e-llm-inference-service] routes=[ROUTER_ROUTES[2], ROUTER_ROUTES[3]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-dp-ep-gpu", [e2e-llm-inference-service] "workload-dp-ep-prefill-gpu", [e2e-llm-inference-service] "model-deepseek-v2-lite", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="Delve into the multifaceted implications of a fully disaggregated cloud architecture, specifically " [e2e-llm-inference-service] "where the compute plane (P) and the data plane (D) are independently deployed and managed for a " [e2e-llm-inference-service] "geographically distributed, high-throughput, low-latency microservices ecosystem. Beyond the " [e2e-llm-inference-service] "fundamental challenges of network latency and data consistency, elaborate on the advanced " [e2e-llm-inference-service] "considerations and trade-offs inherent in such a setup: 1. Network Architecture and Protocols: " [e2e-llm-inference-service] "How would the network fabric and underlying protocols (e.g., RDMA, custom transport layers) need to " [e2e-llm-inference-service] "evolve to support optimal performance and minimize inter-plane communication overhead, especially for " [e2e-llm-inference-service] "synchronous operations? Discuss the role of network programmability (e.g., SDN, P4) in dynamically " [e2e-llm-inference-service] "optimizing routing and traffic flow between P and D. 2. Advanced Data Consistency and Durability: " [e2e-llm-inference-service] "Explore sophisticated data consistency models (e.g., causal consistency, strong eventual consistency) " [e2e-llm-inference-service] "and their applicability in balancing performance and data integrity across a globally distributed data plane. " [e2e-llm-inference-service] "Detail strategies for ensuring data durability and fault tolerance, including multi-region replication, " [e2e-llm-inference-service] "intelligent partitioning, and recovery mechanisms in the event of partial or full plane failures. " [e2e-llm-inference-service] "3. Dynamic Resource Orchestration and Cost Optimization: Analyze how an orchestration layer would intelligently " [e2e-llm-inference-service] "manage the independent scaling of compute (P) and data (D) resources, considering fluctuating workloads, " [e2e-llm-inference-service] "cost efficiency, and performance targets (e.g., using predictive analytics for resource provisioning). " [e2e-llm-inference-service] "Discuss mechanisms for dynamically reallocating compute nodes to different data partitions based on " [e2e-llm-inference-service] "workload patterns and data locality, potentially involving live migration strategies. " [e2e-llm-inference-service] "4. Security and Compliance in a Distributed Landscape: Address the enhanced security perimeter " [e2e-llm-inference-service] "challenges, including securing communication channels between P and D (encryption in transit, mutual TLS), " [e2e-llm-inference-service] "fine-grained access control to data at rest and in motion, and identity management across disaggregated " [e2e-llm-inference-service] "components. Discuss how such an architecture impacts compliance with regulatory frameworks (e.g., GDPR, HIPAA) " [e2e-llm-inference-service] "concerning data sovereignty, privacy, and auditability. 5. Operational Complexity and Observability: " [e2e-llm-inference-service] "Examine the increased complexity in monitoring, logging, and tracing across highly decoupled compute and " [e2e-llm-inference-service] "data planes. What specialized tooling and practices (e.g., distributed tracing with OpenTelemetry, advanced AIOps) " [e2e-llm-inference-service] "would be essential? How would incident response and troubleshooting differ in this disaggregated environment " [e2e-llm-inference-service] "compared to traditional integrated systems? Consider the challenges of pinpointing root causes across " [e2e-llm-inference-service] "independent failures. 6. Real-world Applicability and Future Trends: Identify specific industries " [e2e-llm-inference-service] "or use cases (e.g., high-frequency trading, IoT edge processing, large language model inference) " [e2e-llm-inference-service] "where the benefits of P/D disaggregation would strongly outweigh its complexities. " [e2e-llm-inference-service] "Conclude by speculating on emerging technologies or paradigms (e.g., serverless compute functions " [e2e-llm-inference-service] "directly interacting with object storage, in-memory disaggregation) that could further drive or " [e2e-llm-inference-service] "transform P/D disaggregation in cloud computing.", [e2e-llm-inference-service] max_tokens=2000, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_gpu, [e2e-llm-inference-service] pytest.mark.cluster_nvidia, [e2e-llm-inference-service] pytest.mark.cluster_nvidia_roce, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-no-scheduler", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="What is KServe?", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.no_scheduler, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-simulated-dp-ep-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="This test simulates DP+EP that can run on CPU, the idea is to test the LWS-based deployment, " [e2e-llm-inference-service] "but without the resources requirements for DP+EP (GPUs and ROCe/IB).", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_multi_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Scheduler config tests [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-inline-config", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-inline-config-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-configmap-ref", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-configmap-ref-test", [e2e-llm-inference-service] before_test=[create_scheduler_configmap], [e2e-llm-inference-service] after_test=[delete_scheduler_configmap], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-replicas", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-ha-replicas-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Precise prefix KV cache routing test [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-precise-prefix-cache-inline-config", [e2e-llm-inference-service] "workload-llmd-simulator-kvcache", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="precise-prefix-cache-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] indirect=["test_case"], [e2e-llm-inference-service] ids=generate_test_id, [e2e-llm-inference-service] ) [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def test_llm_inference_service(test_case: TestCase): # noqa: F811 [e2e-llm-inference-service] inject_k8s_proxy() [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = KServeClient( [e2e-llm-inference-service] config_file=os.environ.get("KUBECONFIG", "~/.kube/config"), [e2e-llm-inference-service] client_configuration=client.Configuration(), [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] service_name = test_case.llm_service.metadata.name [e2e-llm-inference-service] if not test_case.llm_service.metadata.annotations: [e2e-llm-inference-service] test_case.llm_service.metadata.annotations = {} [e2e-llm-inference-service] [e2e-llm-inference-service] test_case.llm_service.metadata.annotations[ [e2e-llm-inference-service] "security.opendatahub.io/enable-auth" [e2e-llm-inference-service] ] = "false" [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] create_llmisvc(kserve_client, test_case.llm_service) [e2e-llm-inference-service] > wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client, test_case.llm_service, test_case.wait_timeout [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:410: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] args = (, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kin...hedul-1402139b'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-schedul-609398f0'}]}, [e2e-llm-inference-service] 'status': None}, 900) [e2e-llm-inference-service] kwargs = {}, func_name = 'wait_for_llm_isvc_ready' [e2e-llm-inference-service] timestamp_start = '2026-04-24T21:30:13.930000', start_time = 1777066213.9302592 [e2e-llm-inference-service] duration = 900.9362668991089, timestamp_end = '2026-04-24T21:45:14.866528' [e2e-llm-inference-service] [e2e-llm-inference-service] @functools.wraps(func) [e2e-llm-inference-service] def wrapper(*args, **kwargs): [e2e-llm-inference-service] func_name = func.__name__ [e2e-llm-inference-service] [e2e-llm-inference-service] timestamp_start = datetime.now().isoformat() [e2e-llm-inference-service] logger.info( [e2e-llm-inference-service] f"[{func_name}] [{timestamp_start}] start - args={args}, kwargs={kwargs}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] start_time = time.time() [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > result = func(*args, **kwargs) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/logging.py:40: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] given = {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security....cas-schedul-1402139b'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-schedul-609398f0'}]}, [e2e-llm-inference-service] 'status': None} [e2e-llm-inference-service] timeout_seconds = 900 [e2e-llm-inference-service] [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client: KServeClient, [e2e-llm-inference-service] given: V1alpha1LLMInferenceService, [e2e-llm-inference-service] timeout_seconds: int = 900, [e2e-llm-inference-service] ) -> str: [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] return True [e2e-llm-inference-service] [e2e-llm-inference-service] > return wait_for(assert_llm_isvc_ready, timeout=timeout_seconds, interval=1.0) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:618: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] assertion_fn = .assert_llm_isvc_ready at 0x7f0805cc9da0> [e2e-llm-inference-service] timeout = 900, interval = 1.0 [e2e-llm-inference-service] [e2e-llm-inference-service] def wait_for( [e2e-llm-inference-service] assertion_fn: Callable[[], Any], timeout: float = 5.0, interval: float = 0.1 [e2e-llm-inference-service] ) -> Any: [e2e-llm-inference-service] """Wait for the assertion to succeed within timeout.""" [e2e-llm-inference-service] deadline = time.time() + timeout [e2e-llm-inference-service] while True: [e2e-llm-inference-service] try: [e2e-llm-inference-service] > return assertion_fn() [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:628: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] > raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] E AssertionError: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:613: AssertionError [e2e-llm-inference-service] ------------------------------ Captured log setup ------------------------------ [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1107 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1056 Checking LLMInferenceServiceConfig router-managed-scheduler-ha-rep-bf59a7b5 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1082 Resource not found, creating LLMInferenceServiceConfig router-managed-scheduler-ha-rep-bf59a7b5 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1092 ✓ Successfully created LLMInferenceServiceConfig router-managed-scheduler-ha-rep-bf59a7b5 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1056 Checking LLMInferenceServiceConfig scheduler-with-replicas-schedul-1402139b in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1082 Resource not found, creating LLMInferenceServiceConfig scheduler-with-replicas-schedul-1402139b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1092 ✓ Successfully created LLMInferenceServiceConfig scheduler-with-replicas-schedul-1402139b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1056 Checking LLMInferenceServiceConfig workload-llmd-simulator-schedul-609398f0 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1082 Resource not found, creating LLMInferenceServiceConfig workload-llmd-simulator-schedul-609398f0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1092 ✓ Successfully created LLMInferenceServiceConfig workload-llmd-simulator-schedul-609398f0 [e2e-llm-inference-service] ------------------------------ Captured log call ------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [test_llm_inference_service] [2026-04-24T21:30:13.847870] start - args=(), kwargs={'test_case': TestCase(base_refs=['router-managed', 'scheduler-with-replicas', 'workload-llmd-simulator'], prompt='KServe is a', service_name='scheduler-ha-replicas-test', endpoint='/v1/completions', max_tokens=100, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, before_test=[], after_test=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-ha-replicas-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-ha-rep-bf59a7b5'}, [e2e-llm-inference-service] {'name': 'scheduler-with-replicas-schedul-1402139b'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-schedul-609398f0'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m')} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1107 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [create_llmisvc] [2026-04-24T21:30:13.860454] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-ha-replicas-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-ha-rep-bf59a7b5'}, [e2e-llm-inference-service] {'name': 'scheduler-with-replicas-schedul-1402139b'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-schedul-609398f0'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [create_llmisvc] [2026-04-24T21:30:13.929915] end - ✅ in 0.069s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_llm_isvc_ready] [2026-04-24T21:30:13.930000] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-ha-replicas-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-ha-rep-bf59a7b5'}, [e2e-llm-inference-service] {'name': 'scheduler-with-replicas-schedul-1402139b'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-schedul-609398f0'}]}, [e2e-llm-inference-service] 'status': None}, 900), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:13.930264] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:13.935269] end - ✅ in 0.005s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:14.935540] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:14.945060] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:15.945449] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:15.953243] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:16.953481] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:16.960426] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:17.960779] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:18.026555] end - ✅ in 0.065s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:19.026903] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:19.124852] end - ✅ in 0.098s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:20.125386] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:20.133944] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:21.134464] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:21.142372] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:22.142644] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:22.150104] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:23.150425] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:23.223993] end - ✅ in 0.073s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:24.224533] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:24.232733] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:25.233164] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:25.241805] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:26.242199] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:26.250301] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:27.250742] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:27.258893] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:28.259344] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:28.269652] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:29.269922] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:29.277641] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:30.277914] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:30.286474] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:31.286854] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:31.295264] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:32.295696] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:32.303951] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:33.304222] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:33.312517] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:34.312928] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:34.321070] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:35.321401] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:35.329506] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:36.329953] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:36.337923] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:37.338216] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:37.347279] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:38.347651] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:38.355757] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:39.356198] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:39.364308] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:40.364713] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:40.373029] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:41.373367] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:41.382515] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:42.382984] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:42.391041] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:43.391367] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:43.399242] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:44.399513] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:44.407338] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:45.407649] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:45.415520] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:46.415911] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:46.424689] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:47.425167] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:47.433358] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:48.433629] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:48.441503] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:49.441875] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:49.450233] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:50.450573] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:50.463985] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:51.464461] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:51.471492] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:52.471771] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:52.479471] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:53.479682] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:53.486612] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:54.486941] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:54.494324] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:55.494703] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:55.502662] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:56.502930] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:56.511603] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:57.511837] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:57.519308] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:58.519753] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:58.528723] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:30:59.529082] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:30:59.537593] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:00.537921] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:00.546565] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:01.546936] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:01.555503] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:02.555907] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:02.564333] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:03.564761] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:03.573675] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:04.574004] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:04.581891] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:05.582195] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:05.589829] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:06.590091] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:06.598494] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:07.598942] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:07.607301] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:08.607710] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:08.616228] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:09.616624] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:09.624872] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:10.625296] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:10.633586] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:11.634030] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:11.641970] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:12.642341] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:12.649823] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:13.650303] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:13.658290] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:14.658585] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:14.666884] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:15.667388] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:15.675344] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:16.675626] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:16.683272] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:17.683552] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:17.693309] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:18.693604] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:18.701522] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:19.701804] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:19.709369] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:20.709655] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:20.718553] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:21.718834] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:21.727012] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:22.727534] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:22.735581] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:23.735880] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:23.744472] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:24.744760] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:24.752333] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:25.752741] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:25.760509] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:26.760768] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:26.768458] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:27.768821] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:27.776710] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:28.776998] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:28.785542] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:29.785874] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:29.793454] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:30.793784] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:30.802358] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:31.802795] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:31.810833] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:32.811086] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:32.818791] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:33.819074] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:33.827523] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:34.827941] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:34.835625] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:35.836054] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:35.844108] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:36.844365] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:36.852550] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:37.852983] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:37.861750] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:38.862178] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:38.870320] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:39.870616] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:39.882769] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:40.883214] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:40.892319] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:41.892598] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:41.900255] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:42.900519] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:42.910168] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:43.910433] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:43.918366] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:44.918635] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:44.926228] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:45.926510] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:45.934442] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:46.934708] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:46.943363] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:47.943623] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:47.952110] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:48.952551] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:48.960259] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:49.960557] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:49.968187] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:50.968522] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:50.983105] end - ✅ in 0.014s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:51.983586] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:51.991759] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:52.992180] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:52.999705] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:54.000007] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:54.008143] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:55.008440] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:55.028229] end - ✅ in 0.020s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:56.028687] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:56.036806] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:57.037221] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:57.045273] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:58.045727] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:58.054530] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:31:59.054861] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:31:59.064746] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:00.065036] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:00.072841] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:01.073183] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:01.081273] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:02.081724] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:02.092038] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:03.092381] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:03.100383] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:04.100815] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:04.109627] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:05.110007] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:05.117430] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:06.117833] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:06.125963] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:07.126247] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:07.134004] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:08.134296] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:08.141978] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:09.142268] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:09.150253] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:10.150537] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:10.158497] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:11.158894] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:11.166475] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:12.166765] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:12.174772] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:13.175232] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:13.185434] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:14.185827] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:14.193809] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:15.194315] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:15.202168] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:16.202413] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:16.210384] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:17.210650] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:17.218746] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:18.219203] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:18.227772] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:19.228222] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:19.236398] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:20.236638] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:20.244494] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:21.244802] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:21.253238] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:22.253620] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:22.261937] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:23.262444] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:23.270349] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:24.270718] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:24.278573] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:25.278856] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:25.287565] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:26.287797] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:26.295651] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:27.295974] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:27.303708] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:28.303982] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:28.311965] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:29.312433] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:29.320752] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:30.321036] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:30.330866] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:31.331182] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:31.339019] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:32.339459] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:32.347344] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:33.347816] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:33.356331] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:34.356756] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:34.364844] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:35.365206] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:35.373846] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:36.374165] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:36.383294] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:37.383690] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:37.391674] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:38.392101] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:38.400911] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:39.401220] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:39.409271] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:40.409570] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:40.417787] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:41.418214] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:41.425990] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:42.426351] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:42.434260] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:43.434488] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:43.442671] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:44.443112] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:44.451178] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:45.451474] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:45.459791] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:46.460228] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:46.468260] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:47.468532] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:47.477957] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:48.478441] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:48.486466] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:49.486804] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:49.493873] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:50.494376] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:50.501496] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:51.501783] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:51.509314] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:52.509600] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:52.517323] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:53.517689] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:53.525625] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:54.525897] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:54.533649] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:55.533916] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:55.541862] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:56.542191] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:56.554009] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:57.554357] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:57.562064] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:58.562401] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:58.570529] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:32:59.570920] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:32:59.579271] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:00.579737] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:00.587936] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:01.588216] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:01.595908] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:02.596175] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:02.603935] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:03.604228] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:03.612559] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:04.612943] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:04.620858] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:05.621203] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:05.630738] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:06.631177] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:06.639564] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:07.640019] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:07.647498] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:08.647848] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:08.657042] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:09.657343] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:09.665658] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:10.666164] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:10.674243] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:11.674527] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:11.682315] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:12.682586] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:12.725764] end - ✅ in 0.043s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:13.726185] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:13.735019] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:14.735396] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:14.824882] end - ✅ in 0.089s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:15.825385] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:15.833677] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:16.834156] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:16.923734] end - ✅ in 0.089s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:17.924006] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:17.932368] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:18.932646] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:18.943895] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:19.944400] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:19.953012] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:20.953336] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:20.963290] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:21.963654] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:21.972398] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:22.972702] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:22.981590] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:23.981840] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:23.990027] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:24.990556] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:24.998637] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:25.998904] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:26.007676] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:27.007935] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:27.032269] end - ✅ in 0.024s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:28.032794] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:28.124737] end - ✅ in 0.092s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:29.125015] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:29.133224] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:30.133680] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:30.146570] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:31.146954] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:31.155813] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:32.156057] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:32.163929] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:33.164174] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:33.172431] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:34.172881] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:34.182470] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:35.182770] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:35.191693] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:36.192108] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:36.200756] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:37.201116] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:37.209111] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:38.209395] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:38.217764] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:39.218196] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:39.226900] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:40.227301] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:40.239242] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:41.239519] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:41.247971] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:42.248406] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:42.257697] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:43.258154] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:43.267290] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:44.267521] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:44.275592] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:45.276038] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:45.284719] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:46.285215] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:46.293444] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:47.293770] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:47.303577] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:48.303969] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:48.312030] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:49.312501] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:49.320506] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:50.320758] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:50.329503] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:51.329777] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:51.337971] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:52.338490] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:52.347542] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:53.347787] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:53.356279] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:54.356696] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:54.365232] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:55.365689] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:55.374593] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:56.374884] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:56.385070] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:57.385385] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:57.393754] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:58.394024] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:58.402488] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:33:59.402877] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:33:59.411060] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:00.411501] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:00.425533] end - ✅ in 0.014s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:01.425919] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:01.434795] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:02.435074] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:02.443564] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:03.443998] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:03.456527] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:04.456822] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:04.465651] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:05.466067] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:05.474629] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:06.475157] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:06.483466] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:07.483921] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:07.492362] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:08.492757] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:08.501000] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:09.501301] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:09.509220] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:10.509500] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:10.517698] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:11.518211] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:11.526589] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:12.527020] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:12.535279] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:13.535541] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:13.544653] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:14.545108] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:14.553481] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:15.553787] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:15.561897] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:16.562403] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:16.573745] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:17.574193] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:17.583175] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:18.583540] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:18.591078] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:19.591392] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:19.599919] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:20.600397] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:20.608061] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:21.608580] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:21.617440] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:22.617742] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:22.626411] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:23.626751] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:23.635349] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:24.635736] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:24.644299] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:25.644817] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:25.652967] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:26.653411] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:26.661505] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:27.661871] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:27.670055] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:28.670560] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:28.678992] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:29.679413] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:29.687281] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:30.687708] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:30.695716] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:31.696036] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:31.703825] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:32.704072] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:32.712468] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:33.712871] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:33.722458] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:34.722891] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:34.731535] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:35.731976] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:35.740984] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:36.741264] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:36.749073] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:37.749358] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:37.757492] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:38.757788] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:38.766156] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:39.766399] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:39.773776] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:40.774092] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:40.782548] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:41.782947] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:41.790784] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:42.791240] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:42.799597] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:43.800043] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:43.810278] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:44.810604] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:44.818771] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:45.819224] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:45.827930] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:46.828447] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:46.837272] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:47.837572] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:47.846063] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:48.846548] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:48.855589] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:49.855980] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:49.864492] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:50.864774] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:50.872913] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:51.873461] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:51.882208] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:52.882508] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:52.891654] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:53.891968] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:53.900409] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:54.900748] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:54.915689] end - ✅ in 0.015s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:55.915977] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:55.925407] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:56.925806] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:56.934252] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:57.934713] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:57.943586] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:58.943942] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:58.952625] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:34:59.952975] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:34:59.961105] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:00.961410] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:00.969359] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:01.969822] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:01.978108] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:02.978441] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:02.987490] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:03.988015] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:03.998628] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:04.999039] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:05.007328] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:06.007647] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:06.017012] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:07.017328] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:07.026178] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:08.026466] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:08.035067] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:09.035360] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:09.042985] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:10.043286] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:10.051493] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:11.051797] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:11.061926] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:12.062345] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:12.070845] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:13.071172] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:13.125245] end - ✅ in 0.054s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:14.125552] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:14.224355] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:15.224622] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:15.234740] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:16.235040] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:16.424500] end - ✅ in 0.189s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:17.424819] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:17.523783] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:18.524185] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:18.531302] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:19.531593] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:19.539216] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:20.539579] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:20.624485] end - ✅ in 0.085s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:21.624812] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:21.723725] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:22.724052] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:22.824608] end - ✅ in 0.100s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:23.824969] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:23.832636] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:24.832910] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:24.846704] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:25.846982] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:25.856394] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:26.856707] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:26.924013] end - ✅ in 0.067s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:27.924442] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:27.933635] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:28.933940] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:28.941869] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:29.942143] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:29.949934] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:30.950215] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:30.957775] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:31.958059] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:31.965855] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:32.966173] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:33.024906] end - ✅ in 0.058s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:34.025191] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:34.033382] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:35.033836] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:35.041868] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:36.042198] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:36.050099] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:37.050403] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:37.058149] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:38.058407] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:38.066222] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:39.066515] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:39.074779] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:40.075091] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:40.085154] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:41.085445] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:41.092680] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:42.093010] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:42.100411] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:43.100765] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:43.108307] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:44.108698] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:44.116573] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:45.116989] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:45.125037] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:46.125546] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:46.133400] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:47.133661] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:47.147867] end - ✅ in 0.014s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:48.148376] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:48.156192] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:49.156575] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:49.164358] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:50.164656] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:50.172905] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:51.173406] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:51.181656] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:52.181929] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:52.190770] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:53.191037] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:53.198597] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:54.198893] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:54.207294] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:55.207735] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:55.215252] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:56.215552] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:56.224177] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:57.224662] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:57.232728] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:58.232999] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:58.242825] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:35:59.243086] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:35:59.251765] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:00.252039] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:00.259800] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:01.260072] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:01.267723] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:02.268152] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:02.275903] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:03.276173] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:03.284332] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:04.284617] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:04.292219] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:05.292555] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:05.301186] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:06.301524] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:06.310417] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:07.310712] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:07.318911] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:08.319192] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:08.332782] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:09.335777] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:09.344174] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:10.344505] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:10.352770] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:11.353302] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:11.361872] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:12.362302] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:12.370382] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:13.370641] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:13.378737] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:14.379198] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:14.387496] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:15.387839] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:15.396335] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:16.396602] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:16.405806] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:17.406318] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:17.414517] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:18.414786] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:18.422554] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:19.422820] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:19.430905] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:20.431200] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:20.439330] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:21.439691] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:21.447807] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:22.448306] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:22.457228] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:23.457667] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:23.465692] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:24.465972] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:24.474424] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:25.474798] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:25.482739] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:26.483009] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:26.493380] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:27.493844] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:27.501730] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:28.501999] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:28.509676] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:29.510004] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:29.517920] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:30.518202] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:30.526484] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:31.526951] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:31.534857] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:32.535113] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:32.543449] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:33.543925] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:33.585447] end - ✅ in 0.041s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:34.585739] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:34.603648] end - ✅ in 0.018s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:35.604075] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:35.612632] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:36.613115] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:36.621745] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:37.622196] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:37.630094] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:38.630386] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:38.638307] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:39.638590] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:39.646287] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:40.646666] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:40.654402] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:41.654680] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:41.663222] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:42.663614] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:42.671423] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:43.671753] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:43.679967] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:44.680441] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:44.688269] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:45.688595] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:45.696362] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:46.696631] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:46.704367] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:47.704631] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:47.712316] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:48.712609] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:48.720431] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:49.720686] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:49.728353] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:50.728590] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:50.736172] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:51.736495] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:51.744786] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:52.745210] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:52.752896] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:53.753174] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:53.760681] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:54.760961] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:54.769835] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:55.770114] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:55.778809] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:56.779075] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:56.786682] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:57.786923] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:57.794808] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:58.795076] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:58.802890] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:36:59.803239] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:36:59.810589] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:00.810954] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:00.818518] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:01.818956] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:01.826957] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:02.827222] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:02.835970] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:03.836234] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:03.924074] end - ✅ in 0.088s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:04.924364] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:05.024673] end - ✅ in 0.100s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:06.025118] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:06.033633] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:07.034092] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:07.124887] end - ✅ in 0.090s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:08.125318] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:08.224918] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:09.225330] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:09.232864] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:10.233296] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:10.241038] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:11.241409] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:11.249363] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:12.249635] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:12.324179] end - ✅ in 0.074s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:13.324447] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:13.332505] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:14.332905] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:14.424431] end - ✅ in 0.091s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:15.424792] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:15.433691] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:16.434110] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:16.441182] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:17.441558] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:17.449656] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:18.449944] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:18.457520] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:19.457833] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:19.465494] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:20.465761] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:20.524538] end - ✅ in 0.058s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:21.524855] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:21.624711] end - ✅ in 0.100s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:22.625183] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:22.635743] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:23.636024] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:23.726039] end - ✅ in 0.090s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:24.726544] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:24.741678] end - ✅ in 0.015s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:25.741950] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:25.751242] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:26.751733] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:26.761026] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:27.761480] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:27.775645] end - ✅ in 0.014s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:28.775972] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:28.784085] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:29.784532] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:29.792367] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:30.792675] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:30.824585] end - ✅ in 0.032s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:31.825015] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:31.833343] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:32.833764] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:32.844211] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:33.844623] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:33.852440] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:34.852876] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:34.862914] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:35.863235] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:35.872069] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:36.872372] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:36.888633] end - ✅ in 0.016s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:37.889102] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:37.897108] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:38.897469] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:38.904997] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:39.905390] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:39.913907] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:40.914377] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:40.934208] end - ✅ in 0.020s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:41.934531] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:41.942716] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:42.943181] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:42.952289] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:43.952607] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:43.963208] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:44.963551] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:44.971834] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:45.972339] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:45.982812] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:46.983330] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:46.991639] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:47.991918] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:48.001073] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:49.001370] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:49.009929] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:50.010465] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:50.018488] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:51.018925] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:51.027196] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:52.027628] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:52.037937] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:53.038433] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:53.046830] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:54.047190] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:54.055667] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:55.055945] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:55.064307] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:56.064781] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:56.073371] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:57.073813] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:57.082324] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:58.082881] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:58.092651] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:37:59.092987] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:37:59.100844] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:00.101188] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:00.109665] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:01.110026] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:01.118634] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:02.119033] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:02.140800] end - ✅ in 0.021s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:03.141177] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:03.149663] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:04.150076] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:04.158237] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:05.158516] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:05.167506] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:06.167968] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:06.176642] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:07.177170] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:07.185794] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:08.186078] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:08.199888] end - ✅ in 0.014s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:09.200295] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:09.211150] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:10.211433] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:10.225199] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:11.225495] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:11.240520] end - ✅ in 0.015s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:12.240926] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:12.249772] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:13.250209] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:13.263692] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:14.264165] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:14.272484] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:15.272927] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:15.281346] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:16.281651] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:16.292834] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:17.293485] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:17.302201] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:18.302597] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:18.310936] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:19.311436] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:19.320216] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:20.320641] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:20.329208] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:21.329664] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:21.338234] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:22.338665] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:22.349044] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:23.349567] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:23.359559] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:24.359862] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:24.368337] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:25.368798] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:25.381631] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:26.381972] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:26.390802] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:27.391089] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:27.398956] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:28.399344] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:28.407185] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:29.407510] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:29.418838] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:30.419169] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:30.427057] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:31.427515] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:31.443508] end - ✅ in 0.016s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:32.443983] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:32.452401] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:33.452875] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:33.462465] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:34.462908] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:34.471081] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:35.471405] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:35.481971] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:36.482369] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:36.493772] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:37.494200] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:37.502296] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:38.502562] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:38.511109] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:39.511414] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:39.520454] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:40.520861] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:40.528983] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:41.529418] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:41.537885] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:42.538358] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:42.548008] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:43.548440] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:43.557237] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:44.557556] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:44.566286] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:45.566588] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:45.575858] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:46.576307] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:46.586979] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:47.587476] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:47.595958] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:48.596289] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:48.605780] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:49.606231] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:49.615113] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:50.615648] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:50.624020] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:51.624565] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:51.633008] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:52.633511] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:52.642046] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:53.642411] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:53.650649] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:54.650946] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:54.659611] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:55.660022] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:55.668751] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:56.669016] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:56.676950] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:57.677416] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:57.685699] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:58.686006] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:58.696782] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:38:59.697093] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:38:59.705094] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:00.705532] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:00.713468] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:01.713946] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:01.722113] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:02.722440] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:02.731113] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:03.731421] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:03.739474] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:04.739737] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:04.748879] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:05.749249] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:05.757578] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:06.757874] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:06.766069] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:07.766425] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:07.775012] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:08.775365] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:08.783411] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:09.783789] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:09.791952] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:10.792324] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:10.800797] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:11.801330] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:11.810045] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:12.810457] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:12.820878] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:13.821439] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:13.829664] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:14.830214] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:14.838571] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:15.838840] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:15.847087] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:16.847530] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:16.855624] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:17.855916] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:17.864348] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:18.864628] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:18.872350] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:19.872746] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:19.882258] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:20.882688] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:20.893104] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:21.893435] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:21.901341] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:22.901733] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:22.911859] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:23.912168] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:23.921335] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:24.921670] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:24.930212] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:25.930693] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:25.943237] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:26.943702] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:26.952563] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:27.952910] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:27.962545] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:28.962972] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:28.971583] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:29.971937] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:29.980866] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:30.981176] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:30.989553] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:31.990002] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:31.998806] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:32.999190] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:33.009058] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:34.009417] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:34.018047] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:35.018324] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:35.027580] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:36.027882] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:36.037764] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:37.038177] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:37.046312] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:38.046559] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:38.054908] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:39.055335] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:39.066302] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:40.066580] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:40.083116] end - ✅ in 0.016s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:41.083408] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:41.091270] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:42.091519] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:42.099334] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:43.099633] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:43.110114] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:44.110397] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:44.119206] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:45.119539] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:45.128230] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:46.128568] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:46.139492] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:47.139781] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:47.147851] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:48.148350] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:48.157471] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:49.157740] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:49.165621] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:50.165850] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:50.174542] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:51.174828] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:51.187594] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:52.187880] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:52.200523] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:53.200830] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:53.208425] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:54.208810] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:54.220689] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:55.220972] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:55.229033] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:56.229380] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:56.237639] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:57.237971] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:57.245781] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:58.246072] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:58.254523] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:39:59.254911] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:39:59.263326] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:00.263758] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:00.271542] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:01.271859] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:01.280495] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:02.280963] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:02.288998] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:03.289295] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:03.305939] end - ✅ in 0.016s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:04.306220] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:04.314159] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:05.314441] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:05.322449] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:06.322901] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:06.331094] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:07.331520] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:07.339685] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:08.340178] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:08.429005] end - ✅ in 0.089s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:09.429366] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:09.438109] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:10.438434] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:10.524248] end - ✅ in 0.086s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:11.524562] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:11.535981] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:12.536528] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:12.545049] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:13.545478] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:13.554472] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:14.554735] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:14.563223] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:15.563536] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:15.572317] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:16.572668] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:16.623784] end - ✅ in 0.051s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:17.624080] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:17.634406] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:18.634695] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:18.730905] end - ✅ in 0.096s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:19.731402] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:19.739380] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:20.739651] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:20.748753] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:21.749174] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:21.758370] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:22.758754] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:22.825415] end - ✅ in 0.066s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:23.825801] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:23.835242] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:24.835543] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:24.844203] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:25.844509] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:25.852452] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:26.852753] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:26.861870] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:27.862181] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:27.924357] end - ✅ in 0.062s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:28.924675] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:29.125590] end - ✅ in 0.201s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:30.125999] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:30.224347] end - ✅ in 0.098s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:31.224807] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:31.324282] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:32.324777] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:32.333030] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:33.333329] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:33.342009] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:34.342306] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:34.351444] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:35.351887] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:35.404581] end - ✅ in 0.052s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:36.404853] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:36.412955] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:37.413241] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:37.421045] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:38.421367] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:38.429447] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:39.429874] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:39.437837] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:40.438329] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:40.524240] end - ✅ in 0.086s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:41.524526] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:41.533024] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:42.533437] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:42.541217] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:43.541553] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:43.550102] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:44.550490] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:44.558503] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:45.558750] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:45.568754] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:46.569187] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:46.577847] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:47.578302] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:47.586043] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:48.586317] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:48.594410] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:49.594673] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:49.602075] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:50.602353] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:50.610657] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:51.611038] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:51.619473] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:52.619810] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:52.628672] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:53.628953] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:53.637402] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:54.637825] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:54.646649] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:55.646962] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:55.655837] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:56.656116] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:56.664491] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:57.664898] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:57.673006] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:58.673311] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:58.680791] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:59.681092] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:59.689225] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:00.689658] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:00.698065] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:01.698347] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:01.706043] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:02.706511] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:02.715197] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:03.715593] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:03.723532] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:04.724003] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:04.732232] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:05.732632] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:05.740924] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:06.741307] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:06.750113] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:07.750390] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:07.758287] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:08.758560] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:08.775281] end - ✅ in 0.016s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:09.775558] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:09.783447] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:10.783881] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:10.792274] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:11.792583] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:11.800984] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:12.801469] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:12.809255] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:13.809599] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:13.817458] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:14.817736] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:14.826172] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:15.826584] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:15.834778] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:16.835066] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:16.843569] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:17.843865] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:17.852019] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:18.852499] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:18.860235] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:19.860557] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:19.869277] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:20.869715] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:20.878263] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:21.878728] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:21.887319] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:22.887742] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:22.896144] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:23.896446] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:23.903700] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:24.903949] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:24.911719] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:25.911998] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:25.919563] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:26.919895] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:26.927846] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:27.928195] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:27.936585] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:28.936998] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:28.945623] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:29.946159] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:29.954350] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:30.954689] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:30.962510] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:31.962816] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:31.970775] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:32.971045] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:32.979331] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:33.979737] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:33.988044] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:34.988347] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:34.996533] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:35.996816] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:36.005164] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:37.005484] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:37.013498] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:38.013816] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:38.021493] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:39.021831] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:39.031078] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:40.031351] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:40.039252] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:41.039608] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:41.047468] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:42.047752] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:42.055837] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:43.056190] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:43.064282] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:44.064600] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:44.072969] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:45.073288] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:45.081325] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:46.081613] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:46.089270] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:47.089553] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:47.098111] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:48.098428] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:48.106921] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:49.107510] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:49.115482] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:50.115757] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:50.123522] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:51.123806] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:51.131694] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:52.131973] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:52.139764] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:53.140025] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:53.147957] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:54.148312] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:54.157336] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:55.157740] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:55.166300] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:56.166628] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:56.174600] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:57.174927] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:57.182991] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:58.183281] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:58.190723] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:59.191027] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:59.199638] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:00.199947] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:00.207191] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:01.207532] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:01.215675] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:02.216051] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:02.224538] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:03.224871] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:03.233435] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:04.233776] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:04.241769] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:05.242186] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:05.250201] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:06.250592] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:06.258547] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:07.258908] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:07.266619] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:08.267054] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:08.275368] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:09.275745] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:09.284271] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:10.284586] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:10.292279] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:11.292616] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:11.300152] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:12.300501] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:12.308687] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:13.309012] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:13.323463] end - ✅ in 0.014s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:14.323796] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:14.331663] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:15.331987] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:15.339795] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:16.340116] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:16.348739] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:17.349169] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:17.357240] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:18.357620] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:18.366089] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:19.366583] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:19.374338] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:20.374652] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:20.382527] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:21.382793] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:21.390635] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:22.390931] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:22.399500] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:23.399852] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:23.407730] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:24.408016] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:24.416055] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:25.416402] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:25.424481] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:26.424853] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:26.433868] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:27.434231] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:27.442841] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:28.443110] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:28.451388] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:29.451836] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:29.459873] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:30.460181] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:30.468263] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:31.468668] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:31.477544] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:32.477886] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:32.485800] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:33.486194] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:33.494258] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:34.494646] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:34.502390] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:35.502746] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:35.510741] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:36.511040] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:36.520292] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:37.520663] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:37.528356] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:38.528634] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:38.537266] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:39.537639] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:39.545279] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:40.545613] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:40.553531] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:41.553857] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:41.561631] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:42.561965] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:42.569360] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:43.569652] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:43.578278] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:44.578608] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:44.586306] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:45.586606] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:45.594469] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:46.594851] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:46.602503] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:47.602825] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:47.613617] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:48.613891] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:48.621666] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:49.621957] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:49.629526] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:50.629843] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:50.637835] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:51.638114] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:51.645905] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:52.646252] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:52.654535] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:53.654934] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:53.662926] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:54.663391] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:54.671339] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:55.671719] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:55.679307] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:56.679644] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:56.687840] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:57.688188] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:57.696346] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:58.696779] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:58.704690] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:59.704990] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:59.712814] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:00.713152] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:00.720793] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:01.721173] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:01.728866] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:02.729175] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:02.737795] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:03.738182] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:03.745835] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:04.746149] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:04.753607] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:05.754172] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:05.762149] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:06.762514] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:06.770643] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:07.771207] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:07.779278] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:08.779668] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:08.787313] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:09.787578] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:09.795395] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:10.795732] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:10.804589] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:11.804954] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:11.813025] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:12.813501] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:12.823517] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:13.823826] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:13.831584] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:14.831885] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:14.840243] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:15.840581] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:15.849299] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:16.849628] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:16.858382] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:17.858885] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:17.867682] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:18.868010] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:18.876617] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:19.876908] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:19.884533] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:20.884810] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:20.893485] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:21.893797] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:21.901838] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:22.902210] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:22.911519] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:23.911917] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:23.920505] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:24.920963] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:24.932801] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:25.933071] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:25.941374] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:26.941894] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:26.953262] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:27.953688] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:27.962177] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:28.962677] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:28.971178] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:29.971592] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:29.981945] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:30.982305] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:30.992819] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:31.993206] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:32.001486] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:33.001876] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:33.011065] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:34.011520] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:34.020776] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:35.021057] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:35.029348] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:36.029696] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:36.037920] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:37.038333] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:37.046215] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:38.046469] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:38.054294] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:39.054609] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:39.063641] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:40.064087] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:40.071941] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:41.072413] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:41.080395] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:42.080680] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:42.088842] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:43.089149] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:43.096951] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:44.097519] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:44.105080] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:45.105481] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:45.113850] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:46.114199] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:46.122405] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:47.122718] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:47.131192] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:48.131661] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:48.139628] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:49.139915] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:49.148546] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:50.149023] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:50.158431] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:51.158893] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:51.167459] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:52.167887] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:52.176609] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:53.176889] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:53.184728] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:54.185017] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:54.193429] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:55.193806] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:55.201972] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:56.202301] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:56.210880] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:57.211162] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:57.219017] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:58.219313] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:58.227804] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:59.228197] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:59.236627] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:00.237088] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:00.245044] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:01.245345] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:01.253425] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:02.253738] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:02.261759] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:03.262177] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:03.270242] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:04.270543] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:04.278219] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:05.278551] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:05.286245] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:06.286545] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:06.294407] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:07.294853] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:07.302231] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:08.302530] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:08.310386] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:09.310666] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:09.318519] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:10.318771] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:10.326604] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:11.326905] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:11.334578] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:12.334905] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:12.342916] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:13.343222] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:13.351113] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:14.351428] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:14.359572] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:15.359832] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:15.367569] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:16.367900] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:16.376571] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:17.376859] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:17.385284] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:18.385725] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:18.394282] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:19.394716] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:19.403279] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:20.403743] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:20.411671] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:21.411966] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:21.419947] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:22.420509] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:22.428899] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:23.429242] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:23.437564] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:24.437934] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:24.445330] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:25.445655] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:25.453790] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:26.454151] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:26.462597] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:27.463010] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:27.470626] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:28.470983] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:28.479741] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:29.480114] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:29.488604] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:30.489040] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:30.497250] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:31.497655] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:31.506164] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:32.506454] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:32.514017] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:33.514365] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:33.523162] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:34.523458] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:34.531609] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:35.532078] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:35.540832] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:36.541374] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:36.550974] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:37.551284] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:37.559222] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:38.559525] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:38.567449] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:39.567784] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:39.576112] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:40.576519] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:40.584090] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:41.584500] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:41.593055] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:42.593421] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:42.601523] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:43.601854] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:43.609789] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:44.610096] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:44.618439] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:45.618739] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:45.627050] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:46.627566] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:46.635674] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:47.636004] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:47.644243] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:48.644624] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:48.652235] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:49.652592] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:49.660849] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:50.661205] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:50.669536] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:51.669878] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:51.678421] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:52.678820] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:52.686466] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:53.686735] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:53.694333] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:54.694753] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:54.702438] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:55.702776] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:55.710822] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:56.711102] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:56.718987] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:57.719299] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:57.727335] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:58.727659] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:58.735555] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:59.735999] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:59.743624] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:00.744002] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:00.753068] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:01.753420] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:01.760851] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:02.761204] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:02.769489] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:03.769935] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:03.777798] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:04.778146] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:04.785864] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:05.786198] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:05.793755] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:06.794017] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:06.801770] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:07.802088] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:07.810312] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:08.810649] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:08.818065] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:09.818417] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:09.826287] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:10.826610] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:10.834568] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:11.834930] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:11.842689] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:12.842961] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:12.850187] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:13.850468] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:13.858036] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:14.858380] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:14.866346] end - ✅ in 0.008s [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [wait_for_llm_isvc_ready] [2026-04-24T21:45:14.866528] end - ❌ 900.936s: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:14.866629] start - args=(, 'scheduler-ha-replicas-test', 'kserve-ci-e2e-test'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:14.873009] end - ✅ in 0.006s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [delete_llmisvc] [2026-04-24T21:45:16.568003] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-ha-replicas-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-ha-rep-bf59a7b5'}, [e2e-llm-inference-service] {'name': 'scheduler-with-replicas-schedul-1402139b'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-schedul-609398f0'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [delete_llmisvc] [2026-04-24T21:45:16.592618] end - ✅ in 0.024s [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [test_llm_inference_service] [2026-04-24T21:45:16.592691] end - ❌ 902.744s: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:18Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:30:49Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:30:25Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] _ test_llm_inference_service[router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] _ [e2e-llm-inference-service] [gw0] linux -- Python 3.11.13 /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-custom-route-timeout-pd', 'scheduler-managed', 'workload-pd-cpu', 'model-fb-opt-125m'], pr... {'name': 'model-fb-opt-125m-custom-route-e92a9ea6'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] @pytest.mark.parametrize( [e2e-llm-inference-service] "test_case", [e2e-llm-inference-service] [ [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-gateway-ref", [e2e-llm-inference-service] "router-with-managed-route", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/completions", [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=create_response_assertion(with_field="choices"), [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[0]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-custom-route-timeout", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="custom-route-timeout-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-refs", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="router-with-refs-test", [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[0]], [e2e-llm-inference-service] routes=[ROUTER_ROUTES[0], ROUTER_ROUTES[1]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=["router-managed", "workload-pd-cpu", "model-fb-opt-125m"], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-custom-route-timeout-pd", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-pd-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] service_name="custom-route-timeout-pd-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-refs-pd", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-pd-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] service_name="router-with-refs-pd-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[1]], [e2e-llm-inference-service] routes=[ROUTER_ROUTES[2], ROUTER_ROUTES[3]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-dp-ep-gpu", [e2e-llm-inference-service] "workload-dp-ep-prefill-gpu", [e2e-llm-inference-service] "model-deepseek-v2-lite", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="Delve into the multifaceted implications of a fully disaggregated cloud architecture, specifically " [e2e-llm-inference-service] "where the compute plane (P) and the data plane (D) are independently deployed and managed for a " [e2e-llm-inference-service] "geographically distributed, high-throughput, low-latency microservices ecosystem. Beyond the " [e2e-llm-inference-service] "fundamental challenges of network latency and data consistency, elaborate on the advanced " [e2e-llm-inference-service] "considerations and trade-offs inherent in such a setup: 1. Network Architecture and Protocols: " [e2e-llm-inference-service] "How would the network fabric and underlying protocols (e.g., RDMA, custom transport layers) need to " [e2e-llm-inference-service] "evolve to support optimal performance and minimize inter-plane communication overhead, especially for " [e2e-llm-inference-service] "synchronous operations? Discuss the role of network programmability (e.g., SDN, P4) in dynamically " [e2e-llm-inference-service] "optimizing routing and traffic flow between P and D. 2. Advanced Data Consistency and Durability: " [e2e-llm-inference-service] "Explore sophisticated data consistency models (e.g., causal consistency, strong eventual consistency) " [e2e-llm-inference-service] "and their applicability in balancing performance and data integrity across a globally distributed data plane. " [e2e-llm-inference-service] "Detail strategies for ensuring data durability and fault tolerance, including multi-region replication, " [e2e-llm-inference-service] "intelligent partitioning, and recovery mechanisms in the event of partial or full plane failures. " [e2e-llm-inference-service] "3. Dynamic Resource Orchestration and Cost Optimization: Analyze how an orchestration layer would intelligently " [e2e-llm-inference-service] "manage the independent scaling of compute (P) and data (D) resources, considering fluctuating workloads, " [e2e-llm-inference-service] "cost efficiency, and performance targets (e.g., using predictive analytics for resource provisioning). " [e2e-llm-inference-service] "Discuss mechanisms for dynamically reallocating compute nodes to different data partitions based on " [e2e-llm-inference-service] "workload patterns and data locality, potentially involving live migration strategies. " [e2e-llm-inference-service] "4. Security and Compliance in a Distributed Landscape: Address the enhanced security perimeter " [e2e-llm-inference-service] "challenges, including securing communication channels between P and D (encryption in transit, mutual TLS), " [e2e-llm-inference-service] "fine-grained access control to data at rest and in motion, and identity management across disaggregated " [e2e-llm-inference-service] "components. Discuss how such an architecture impacts compliance with regulatory frameworks (e.g., GDPR, HIPAA) " [e2e-llm-inference-service] "concerning data sovereignty, privacy, and auditability. 5. Operational Complexity and Observability: " [e2e-llm-inference-service] "Examine the increased complexity in monitoring, logging, and tracing across highly decoupled compute and " [e2e-llm-inference-service] "data planes. What specialized tooling and practices (e.g., distributed tracing with OpenTelemetry, advanced AIOps) " [e2e-llm-inference-service] "would be essential? How would incident response and troubleshooting differ in this disaggregated environment " [e2e-llm-inference-service] "compared to traditional integrated systems? Consider the challenges of pinpointing root causes across " [e2e-llm-inference-service] "independent failures. 6. Real-world Applicability and Future Trends: Identify specific industries " [e2e-llm-inference-service] "or use cases (e.g., high-frequency trading, IoT edge processing, large language model inference) " [e2e-llm-inference-service] "where the benefits of P/D disaggregation would strongly outweigh its complexities. " [e2e-llm-inference-service] "Conclude by speculating on emerging technologies or paradigms (e.g., serverless compute functions " [e2e-llm-inference-service] "directly interacting with object storage, in-memory disaggregation) that could further drive or " [e2e-llm-inference-service] "transform P/D disaggregation in cloud computing.", [e2e-llm-inference-service] max_tokens=2000, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_gpu, [e2e-llm-inference-service] pytest.mark.cluster_nvidia, [e2e-llm-inference-service] pytest.mark.cluster_nvidia_roce, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-no-scheduler", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="What is KServe?", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.no_scheduler, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-simulated-dp-ep-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="This test simulates DP+EP that can run on CPU, the idea is to test the LWS-based deployment, " [e2e-llm-inference-service] "but without the resources requirements for DP+EP (GPUs and ROCe/IB).", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_multi_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Scheduler config tests [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-inline-config", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-inline-config-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-configmap-ref", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-configmap-ref-test", [e2e-llm-inference-service] before_test=[create_scheduler_configmap], [e2e-llm-inference-service] after_test=[delete_scheduler_configmap], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-replicas", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-ha-replicas-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Precise prefix KV cache routing test [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-precise-prefix-cache-inline-config", [e2e-llm-inference-service] "workload-llmd-simulator-kvcache", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="precise-prefix-cache-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] indirect=["test_case"], [e2e-llm-inference-service] ids=generate_test_id, [e2e-llm-inference-service] ) [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def test_llm_inference_service(test_case: TestCase): # noqa: F811 [e2e-llm-inference-service] inject_k8s_proxy() [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = KServeClient( [e2e-llm-inference-service] config_file=os.environ.get("KUBECONFIG", "~/.kube/config"), [e2e-llm-inference-service] client_configuration=client.Configuration(), [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] service_name = test_case.llm_service.metadata.name [e2e-llm-inference-service] if not test_case.llm_service.metadata.annotations: [e2e-llm-inference-service] test_case.llm_service.metadata.annotations = {} [e2e-llm-inference-service] [e2e-llm-inference-service] test_case.llm_service.metadata.annotations[ [e2e-llm-inference-service] "security.opendatahub.io/enable-auth" [e2e-llm-inference-service] ] = "false" [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] create_llmisvc(kserve_client, test_case.llm_service) [e2e-llm-inference-service] > wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client, test_case.llm_service, test_case.wait_timeout [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:410: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] args = (, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kin...ute-ti-2766b49d'}, [e2e-llm-inference-service] {'name': 'model-fb-opt-125m-custom-route-e92a9ea6'}]}, [e2e-llm-inference-service] 'status': None}, 900) [e2e-llm-inference-service] kwargs = {}, func_name = 'wait_for_llm_isvc_ready' [e2e-llm-inference-service] timestamp_start = '2026-04-24T21:40:08.823716', start_time = 1777066808.824188 [e2e-llm-inference-service] duration = 900.952376127243, timestamp_end = '2026-04-24T21:55:09.776564' [e2e-llm-inference-service] [e2e-llm-inference-service] @functools.wraps(func) [e2e-llm-inference-service] def wrapper(*args, **kwargs): [e2e-llm-inference-service] func_name = func.__name__ [e2e-llm-inference-service] [e2e-llm-inference-service] timestamp_start = datetime.now().isoformat() [e2e-llm-inference-service] logger.info( [e2e-llm-inference-service] f"[{func_name}] [{timestamp_start}] start - args={args}, kwargs={kwargs}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] start_time = time.time() [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > result = func(*args, **kwargs) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/logging.py:40: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] given = {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security....tom-route-ti-2766b49d'}, [e2e-llm-inference-service] {'name': 'model-fb-opt-125m-custom-route-e92a9ea6'}]}, [e2e-llm-inference-service] 'status': None} [e2e-llm-inference-service] timeout_seconds = 900 [e2e-llm-inference-service] [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client: KServeClient, [e2e-llm-inference-service] given: V1alpha1LLMInferenceService, [e2e-llm-inference-service] timeout_seconds: int = 900, [e2e-llm-inference-service] ) -> str: [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] return True [e2e-llm-inference-service] [e2e-llm-inference-service] > return wait_for(assert_llm_isvc_ready, timeout=timeout_seconds, interval=1.0) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:618: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] assertion_fn = .assert_llm_isvc_ready at 0x7fc91e8ae840> [e2e-llm-inference-service] timeout = 900, interval = 1.0 [e2e-llm-inference-service] [e2e-llm-inference-service] def wait_for( [e2e-llm-inference-service] assertion_fn: Callable[[], Any], timeout: float = 5.0, interval: float = 0.1 [e2e-llm-inference-service] ) -> Any: [e2e-llm-inference-service] """Wait for the assertion to succeed within timeout.""" [e2e-llm-inference-service] deadline = time.time() + timeout [e2e-llm-inference-service] while True: [e2e-llm-inference-service] try: [e2e-llm-inference-service] > return assertion_fn() [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:628: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] > raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] E AssertionError: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:613: AssertionError [e2e-llm-inference-service] ------------------------------ Captured log setup ------------------------------ [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1107 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1056 Checking LLMInferenceServiceConfig router-custom-route-timeout-pd-8b408578 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1082 Resource not found, creating LLMInferenceServiceConfig router-custom-route-timeout-pd-8b408578 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1092 ✓ Successfully created LLMInferenceServiceConfig router-custom-route-timeout-pd-8b408578 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1056 Checking LLMInferenceServiceConfig scheduler-managed-custom-route-35ef63b2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1082 Resource not found, creating LLMInferenceServiceConfig scheduler-managed-custom-route-35ef63b2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1092 ✓ Successfully created LLMInferenceServiceConfig scheduler-managed-custom-route-35ef63b2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1056 Checking LLMInferenceServiceConfig workload-pd-cpu-custom-route-ti-2766b49d in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1082 Resource not found, creating LLMInferenceServiceConfig workload-pd-cpu-custom-route-ti-2766b49d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1092 ✓ Successfully created LLMInferenceServiceConfig workload-pd-cpu-custom-route-ti-2766b49d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1056 Checking LLMInferenceServiceConfig model-fb-opt-125m-custom-route-e92a9ea6 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1082 Resource not found, creating LLMInferenceServiceConfig model-fb-opt-125m-custom-route-e92a9ea6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1092 ✓ Successfully created LLMInferenceServiceConfig model-fb-opt-125m-custom-route-e92a9ea6 [e2e-llm-inference-service] ------------------------------ Captured log call ------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [test_llm_inference_service] [2026-04-24T21:40:08.737937] start - args=(), kwargs={'test_case': TestCase(base_refs=['router-custom-route-timeout-pd', 'scheduler-managed', 'workload-pd-cpu', 'model-fb-opt-125m'], prompt='You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.', service_name='custom-route-timeout-pd-test', endpoint='/v1/completions', max_tokens=100, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, before_test=[], after_test=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'custom-route-timeout-pd-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-custom-route-timeout-pd-8b408578'}, [e2e-llm-inference-service] {'name': 'scheduler-managed-custom-route-35ef63b2'}, [e2e-llm-inference-service] {'name': 'workload-pd-cpu-custom-route-ti-2766b49d'}, [e2e-llm-inference-service] {'name': 'model-fb-opt-125m-custom-route-e92a9ea6'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m')} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1107 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [create_llmisvc] [2026-04-24T21:40:08.750727] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'custom-route-timeout-pd-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-custom-route-timeout-pd-8b408578'}, [e2e-llm-inference-service] {'name': 'scheduler-managed-custom-route-35ef63b2'}, [e2e-llm-inference-service] {'name': 'workload-pd-cpu-custom-route-ti-2766b49d'}, [e2e-llm-inference-service] {'name': 'model-fb-opt-125m-custom-route-e92a9ea6'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [create_llmisvc] [2026-04-24T21:40:08.823563] end - ✅ in 0.073s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_llm_isvc_ready] [2026-04-24T21:40:08.823716] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'custom-route-timeout-pd-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-custom-route-timeout-pd-8b408578'}, [e2e-llm-inference-service] {'name': 'scheduler-managed-custom-route-35ef63b2'}, [e2e-llm-inference-service] {'name': 'workload-pd-cpu-custom-route-ti-2766b49d'}, [e2e-llm-inference-service] {'name': 'model-fb-opt-125m-custom-route-e92a9ea6'}]}, [e2e-llm-inference-service] 'status': None}, 900), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:08.824192] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:08.832102] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:09.832381] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:09.839742] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:10.840179] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:10.847741] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:11.848115] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:11.856714] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:12.857178] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:12.864707] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:13.865000] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:13.872391] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:14.872632] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:14.924628] end - ✅ in 0.052s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:15.925013] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:15.932730] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:16.933153] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:16.940520] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:17.940822] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:18.024245] end - ✅ in 0.083s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:19.024511] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:19.031492] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:20.031754] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:20.039526] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:21.039799] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:21.047852] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:22.048311] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:22.124034] end - ✅ in 0.075s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:23.124522] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:23.134015] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:24.134477] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:24.143649] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:25.143927] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:25.157515] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:26.157853] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:26.165772] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:27.166029] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:27.325986] end - ✅ in 0.160s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:28.326318] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:28.424890] end - ✅ in 0.098s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:29.425247] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:29.525216] end - ✅ in 0.100s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:30.525628] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:30.624726] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:31.625163] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:31.725770] end - ✅ in 0.100s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:32.726174] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:32.734865] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:33.735184] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:33.743213] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:34.743611] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:34.752642] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:35.753164] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:35.761274] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:36.761679] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:36.770046] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:37.770353] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:37.825596] end - ✅ in 0.055s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:38.825987] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:38.834218] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:39.834460] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:39.842004] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:40.842314] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:40.850366] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:41.850648] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:41.858277] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:42.858549] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:42.866384] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:43.866663] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:43.874591] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:44.875186] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:44.882940] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:45.883475] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:45.891532] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:46.891886] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:46.899988] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:47.900504] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:47.908634] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:48.908959] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:48.917462] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:49.917747] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:49.925921] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:50.926429] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:50.934561] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:51.934834] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:51.943851] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:52.944258] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:52.954717] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:53.954966] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:53.963669] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:54.963941] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:54.972669] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:55.973082] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:55.981481] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:56.981883] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:56.990315] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:57.990836] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:57.999280] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:40:58.999729] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:40:59.016972] end - ✅ in 0.017s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:00.017249] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:00.025935] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:01.026216] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:01.034306] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:02.034542] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:02.042784] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:03.043173] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:03.054850] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:04.055212] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:04.063792] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:05.064181] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:05.072377] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:06.072887] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:06.081300] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:07.081565] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:07.089995] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:08.090366] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:08.098890] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:09.099325] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:09.107823] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:10.108310] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:10.115611] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:11.115922] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:11.123402] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:12.123682] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:12.133206] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:13.133547] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:13.141477] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:14.141722] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:14.149512] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:15.149765] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:15.157820] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:16.158104] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:16.166487] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:17.166725] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:17.174394] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:18.174634] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:18.183170] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:19.183420] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:19.191701] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:20.191989] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:20.200205] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:21.200596] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:21.208796] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:22.209115] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:22.216974] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:23.217319] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:23.224697] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:24.225000] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:24.233301] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:25.233793] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:25.242805] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:26.243218] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:26.252294] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:27.252741] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:27.261008] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:28.261387] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:28.270906] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:29.271211] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:29.279453] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:30.279895] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:30.288213] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:31.288597] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:31.296545] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:32.296845] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:32.305030] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:33.305295] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:33.313564] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:34.313986] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:34.322669] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:35.323113] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:35.331581] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:36.332001] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:36.339840] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:37.340206] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:37.348716] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:38.349174] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:38.357357] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:39.357613] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:39.364840] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:40.365190] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:40.373276] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:41.373557] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:41.381679] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:42.382054] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:42.390229] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:43.390531] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:43.412261] end - ✅ in 0.021s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:44.412631] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:44.420170] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:45.420443] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:45.430100] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:46.430452] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:46.439747] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:47.440167] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:47.448948] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:48.449248] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:48.457113] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:49.457432] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:49.465863] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:50.466170] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:50.474606] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:51.474876] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:51.483729] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:52.484009] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:52.493728] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:53.493989] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:53.501976] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:54.502251] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:54.509649] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:55.509909] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:55.517765] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:56.518278] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:56.526495] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:57.526889] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:57.535285] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:58.535648] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:58.543931] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:41:59.544264] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:41:59.551905] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:00.552189] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:00.560218] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:01.560557] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:01.568164] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:02.568497] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:02.576780] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:03.577340] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:03.588507] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:04.588823] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:04.596745] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:05.597176] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:05.605829] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:06.606328] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:06.614381] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:07.614706] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:07.624000] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:08.624295] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:08.632217] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:09.632508] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:09.640506] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:10.640791] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:10.648970] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:11.649429] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:11.658041] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:12.658520] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:12.667478] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:13.667773] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:13.675117] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:14.675436] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:14.682759] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:15.683017] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:15.712735] end - ✅ in 0.029s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:16.713048] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:16.720861] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:17.721180] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:17.728679] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:18.729031] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:18.736793] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:19.737095] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:19.745613] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:20.745915] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:20.754683] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:21.755001] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:21.762812] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:22.763583] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:22.771692] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:23.772027] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:23.780400] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:24.780843] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:24.788859] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:25.789183] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:25.796964] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:26.797275] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:26.805159] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:27.805485] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:27.813846] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:28.814188] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:28.822456] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:29.822746] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:29.830918] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:30.831216] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:30.839029] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:31.839458] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:31.847482] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:32.847836] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:32.856279] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:33.856745] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:33.864769] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:34.865096] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:34.873202] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:35.873503] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:35.881887] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:36.882285] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:36.892951] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:37.893367] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:37.901346] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:38.901600] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:38.908722] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:39.909033] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:39.917062] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:40.917479] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:40.925606] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:41.926018] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:41.933787] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:42.934118] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:42.943564] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:43.944164] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:43.951982] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:44.952295] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:44.960830] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:45.961260] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:45.968633] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:46.968914] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:46.976251] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:47.976535] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:47.984249] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:48.984574] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:48.992641] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:49.992974] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:50.000903] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:51.001184] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:51.009567] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:52.009806] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:52.017880] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:53.018185] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:53.025817] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:54.026071] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:54.033730] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:55.034030] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:55.041718] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:56.041968] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:56.049484] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:57.049767] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:57.057671] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:58.057937] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:58.071202] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:42:59.071517] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:42:59.079306] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:00.079630] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:00.087426] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:01.088014] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:01.095998] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:02.096289] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:02.104509] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:03.104803] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:03.112764] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:04.113195] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:04.121275] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:05.121641] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:05.129624] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:06.130050] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:06.138742] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:07.139239] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:07.147513] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:08.147925] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:08.156284] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:09.156631] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:09.164373] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:10.164764] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:10.172918] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:11.173305] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:11.181086] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:12.181473] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:12.189861] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:13.190403] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:13.198941] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:14.199253] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:14.207517] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:15.207819] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:15.215760] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:16.216073] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:16.224508] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:17.224798] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:17.233006] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:18.233550] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:18.241624] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:19.241895] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:19.252530] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:20.253143] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:20.261220] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:21.261592] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:21.270200] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:22.270542] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:22.279736] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:23.280071] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:23.290547] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:24.290911] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:24.298905] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:25.299215] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:25.307795] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:26.308180] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:26.316215] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:27.316560] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:27.324756] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:28.325093] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:28.333771] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:29.334161] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:29.342342] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:30.342645] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:30.351544] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:31.351939] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:31.360495] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:32.360877] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:32.376650] end - ✅ in 0.015s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:33.376999] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:33.385383] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:34.385751] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:34.393794] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:35.394223] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:35.402964] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:36.403489] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:36.412338] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:37.412613] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:37.420750] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:38.421205] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:38.431603] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:39.431956] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:39.442234] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:40.442540] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:40.450916] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:41.451388] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:41.459494] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:42.459890] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:42.467759] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:43.468076] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:43.476102] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:44.476476] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:44.484506] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:45.484823] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:45.492760] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:46.493083] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:46.501355] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:47.501729] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:47.510461] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:48.510794] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:48.518754] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:49.519077] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:49.527486] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:50.527854] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:50.535936] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:51.536379] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:51.545031] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:52.545449] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:52.553702] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:53.554157] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:53.562004] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:54.562473] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:54.570736] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:55.571067] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:55.579556] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:56.579972] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:56.588404] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:57.588748] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:57.597419] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:58.597788] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:58.605978] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:43:59.606296] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:43:59.614869] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:00.615192] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:00.623484] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:01.623941] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:01.632337] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:02.632722] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:02.640778] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:03.641204] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:03.650447] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:04.650794] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:04.659152] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:05.659523] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:05.667929] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:06.668296] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:06.677327] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:07.677606] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:07.685487] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:08.685749] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:08.693159] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:09.693445] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:09.701740] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:10.702061] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:10.709716] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:11.710157] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:11.718505] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:12.718784] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:12.727306] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:13.727661] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:13.735720] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:14.736036] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:14.743761] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:15.744044] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:15.751512] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:16.751774] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:16.760544] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:17.760811] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:17.768643] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:18.768968] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:18.776766] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:19.777089] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:19.784982] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:20.785316] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:20.793790] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:21.794073] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:21.801528] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:22.801831] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:22.810199] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:23.810490] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:23.818202] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:24.818531] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:24.826116] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:25.826461] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:25.835279] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:26.835681] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:26.843585] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:27.843886] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:27.851686] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:28.851983] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:28.860667] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:29.861073] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:29.869611] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:30.870074] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:30.877598] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:31.877877] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:31.885431] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:32.885711] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:32.894070] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:33.894373] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:33.902398] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:34.902763] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:34.910320] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:35.910589] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:35.918365] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:36.918613] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:36.926272] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:37.926554] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:37.934407] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:38.934714] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:38.943334] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:39.943724] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:39.952311] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:40.952581] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:40.960431] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:41.960688] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:41.968774] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:42.969239] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:42.977554] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:43.977833] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:43.986311] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:44.986739] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:44.994580] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:45.994862] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:46.003087] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:47.003580] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:47.012430] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:48.012754] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:48.022434] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:49.022720] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:49.031382] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:50.031767] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:50.040227] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:51.040668] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:51.048900] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:52.049336] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:52.057471] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:53.057772] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:53.065673] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:54.066031] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:54.073987] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:55.074355] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:55.081927] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:56.082227] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:56.091534] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:57.091843] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:57.099382] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:58.099709] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:58.107838] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:44:59.108193] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:44:59.116094] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:00.116417] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:00.124292] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:01.124622] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:01.132490] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:02.132800] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:02.140642] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:03.140972] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:03.149566] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:04.149985] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:04.158110] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:05.158394] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:05.166856] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:06.167195] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:06.175257] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:07.175558] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:07.183467] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:08.183716] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:08.191252] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:09.191514] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:09.199913] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:10.200268] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:10.208254] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:11.208558] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:11.216226] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:12.216527] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:12.225307] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:13.225778] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:13.235447] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:14.235697] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:14.243583] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:15.243842] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:15.251645] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:16.251980] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:16.259603] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:17.259895] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:17.324807] end - ✅ in 0.065s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:18.325191] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:18.333746] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:19.334024] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:19.345342] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:20.345620] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:20.353912] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:21.354243] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:21.361930] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:22.362249] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:22.370096] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:23.370453] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:23.378790] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:24.379068] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:24.387604] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:25.387974] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:25.395596] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:26.395992] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:26.404814] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:27.405188] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:27.413954] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:28.414258] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:28.422703] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:29.422999] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:29.431255] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:30.431577] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:30.439843] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:31.440160] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:31.448567] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:32.449010] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:32.457525] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:33.457867] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:33.466025] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:34.466356] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:34.474810] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:35.475317] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:35.484222] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:36.484538] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:36.493782] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:37.494333] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:37.525175] end - ✅ in 0.031s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:38.525488] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:38.624984] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:39.625360] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:39.824564] end - ✅ in 0.199s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:40.824882] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:40.924114] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:41.924460] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:41.933436] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:42.933721] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:42.942748] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:43.943186] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:43.951449] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:44.951695] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:44.959979] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:45.960492] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:45.968991] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:46.969489] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:46.978173] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:47.978635] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:47.987176] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:48.987626] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:48.995654] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:49.995942] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:50.004206] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:51.004649] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:51.024503] end - ✅ in 0.020s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:52.024896] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:52.033835] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:53.034114] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:53.042442] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:54.042893] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:54.050885] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:55.051206] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:55.062460] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:56.062915] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:56.070270] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:57.070599] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:57.080011] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:58.080366] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:58.124165] end - ✅ in 0.044s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:45:59.124603] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:45:59.139182] end - ✅ in 0.014s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:00.139543] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:00.148175] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:01.148484] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:01.157779] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:02.158088] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:02.166700] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:03.167153] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:03.176512] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:04.176812] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:04.184778] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:05.185084] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:05.193184] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:06.193441] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:06.201791] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:07.202074] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:07.210893] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:08.211298] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:08.219672] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:09.219985] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:09.227853] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:10.228190] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:10.326058] end - ✅ in 0.098s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:11.326573] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:11.424385] end - ✅ in 0.098s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:12.424675] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:12.524172] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:13.524488] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:13.624878] end - ✅ in 0.100s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:14.625309] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:14.633659] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:15.633960] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:15.824910] end - ✅ in 0.191s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:16.825423] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:16.832749] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:17.833043] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:17.840981] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:18.841392] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:18.849664] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:19.850147] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:19.858431] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:20.858838] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:20.924707] end - ✅ in 0.066s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:21.925078] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:22.024686] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:23.025068] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:23.124974] end - ✅ in 0.100s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:24.125373] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:24.224964] end - ✅ in 0.099s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:25.225371] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:25.424969] end - ✅ in 0.199s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:26.425344] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:26.525105] end - ✅ in 0.100s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:27.525502] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:27.534169] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:28.534475] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:28.625020] end - ✅ in 0.090s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:29.625419] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:29.633343] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:30.633667] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:30.642018] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:31.642366] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:31.724612] end - ✅ in 0.082s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:32.725113] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:32.733283] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:33.733636] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:33.824158] end - ✅ in 0.090s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:34.824505] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:34.832058] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:35.832324] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:35.840492] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:36.840768] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:36.849030] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:37.849561] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:37.857734] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:38.858083] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:38.865942] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:39.866267] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:39.878382] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:40.878679] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:40.887008] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:41.887352] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:41.895090] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:42.895409] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:42.903922] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:43.904440] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:43.912008] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:44.912346] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:44.920001] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:45.920351] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:45.928181] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:46.928641] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:46.936805] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:47.937159] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:47.944699] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:48.945024] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:48.955645] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:49.956069] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:49.964621] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:50.964990] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:50.973376] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:51.973707] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:51.981836] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:52.982135] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:52.990170] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:53.990437] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:53.998644] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:54.998962] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:55.008423] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:56.008746] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:56.016500] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:57.016811] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:57.024622] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:58.024904] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:58.036443] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:46:59.036758] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:46:59.044749] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:00.045041] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:00.055411] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:01.055838] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:01.064234] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:02.064631] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:02.073412] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:03.073732] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:03.082102] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:04.082420] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:04.092007] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:05.092327] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:05.099874] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:06.100180] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:06.108296] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:07.108571] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:07.116348] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:08.116649] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:08.124777] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:09.125091] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:09.133444] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:10.133717] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:10.141637] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:11.141916] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:11.149991] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:12.150323] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:12.158668] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:13.158994] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:13.167200] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:14.167587] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:14.175790] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:15.176105] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:15.184271] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:16.184586] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:16.192922] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:17.193258] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:17.201571] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:18.201823] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:18.209981] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:19.210296] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:19.218366] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:20.218685] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:20.229497] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:21.229784] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:21.237818] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:22.238172] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:22.246994] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:23.247302] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:23.255960] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:24.256276] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:24.264236] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:25.264604] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:25.273995] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:26.274333] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:26.284520] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:27.284991] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:27.292945] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:28.293223] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:28.301088] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:29.301452] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:29.309245] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:30.309565] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:30.317432] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:31.317747] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:31.326228] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:32.326481] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:32.334358] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:33.334681] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:33.342677] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:34.343000] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:34.351179] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:35.351494] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:35.364243] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:36.364532] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:36.374623] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:37.374920] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:37.383351] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:38.383635] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:38.392187] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:39.392482] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:39.401258] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:40.401564] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:40.409505] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:41.409755] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:41.417716] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:42.418011] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:42.426468] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:43.426723] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:43.435027] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:44.435380] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:44.445310] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:45.445681] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:45.453587] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:46.453812] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:46.461242] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:47.461589] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:47.470443] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:48.470717] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:48.479374] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:49.479761] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:49.524847] end - ✅ in 0.045s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:50.525158] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:50.533740] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:51.534018] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:51.542033] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:52.542372] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:52.550689] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:53.551034] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:53.559191] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:54.559524] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:54.567691] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:55.568060] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:55.576806] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:56.577093] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:56.585355] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:57.585625] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:57.593374] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:58.593807] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:58.601329] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:47:59.601596] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:47:59.609359] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:00.609651] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:00.617782] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:01.618180] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:01.625767] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:02.626091] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:02.634291] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:03.634565] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:03.642062] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:04.642360] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:04.650067] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:05.650364] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:05.659011] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:06.659312] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:06.667276] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:07.667557] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:07.678016] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:08.678351] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:08.686029] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:09.686347] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:09.693713] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:10.694046] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:10.703623] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:11.703914] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:11.713466] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:12.713764] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:12.721616] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:13.721885] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:13.729844] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:14.730148] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:14.738018] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:15.738310] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:15.746271] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:16.746605] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:16.755098] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:17.755381] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:17.763099] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:18.763422] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:18.771609] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:19.771897] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:19.780074] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:20.780414] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:20.788454] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:21.788741] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:21.796783] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:22.797104] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:22.805362] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:23.805683] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:23.813966] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:24.814345] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:24.822738] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:25.823103] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:25.831304] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:26.831724] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:26.840099] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:27.840467] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:27.848791] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:28.849195] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:28.858451] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:29.858791] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:29.866593] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:30.866930] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:30.874916] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:31.875252] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:31.883261] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:32.883528] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:32.895097] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:33.895518] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:33.904245] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:34.904503] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:34.914512] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:35.914824] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:35.923277] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:36.923588] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:36.932441] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:37.932748] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:37.941332] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:38.941611] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:38.950457] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:39.950917] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:39.959925] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:40.960423] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:40.971192] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:41.971523] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:41.980011] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:42.980363] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:42.988880] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:43.989176] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:43.996920] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:44.997181] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:45.005289] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:46.005592] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:46.014982] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:47.015529] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:47.024299] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:48.024620] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:48.032738] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:49.033052] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:49.041743] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:50.041997] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:50.050177] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:51.050567] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:51.059500] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:52.059921] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:52.068027] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:53.068412] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:53.076454] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:54.076766] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:54.086529] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:55.086993] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:55.095223] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:56.095579] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:56.103506] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:57.103817] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:57.112081] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:58.112467] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:58.120506] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:48:59.120776] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:48:59.128833] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:00.129212] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:00.137556] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:01.137976] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:01.155712] end - ✅ in 0.017s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:02.156193] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:02.164506] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:03.164859] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:03.172689] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:04.172982] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:04.181094] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:05.181544] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:05.189331] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:06.189702] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:06.200278] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:07.200576] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:07.208777] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:08.209210] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:08.217671] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:09.218032] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:09.226065] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:10.226529] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:10.233952] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:11.234289] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:11.243046] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:12.243498] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:12.254656] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:13.255145] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:13.267815] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:14.268173] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:14.276038] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:15.276387] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:15.285040] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:16.285602] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:16.293720] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:17.294029] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:17.301905] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:18.302225] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:18.310379] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:19.310799] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:19.319113] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:20.319585] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:20.327885] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:21.328190] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:21.336043] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:22.336410] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:22.344438] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:23.344793] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:23.353635] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:24.354104] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:24.425329] end - ✅ in 0.071s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:25.425754] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:25.434187] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:26.434497] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:26.443450] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:27.443791] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:27.452439] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:28.452897] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:28.461991] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:29.462391] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:29.470193] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:30.470524] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:30.479069] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:31.479398] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:31.487947] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:32.488333] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:32.495763] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:33.496101] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:33.504754] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:34.505310] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:34.513354] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:35.513737] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:35.522647] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:36.523078] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:36.532547] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:37.532912] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:37.540862] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:38.541197] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:38.624553] end - ✅ in 0.083s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:39.624993] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:39.633083] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:40.633583] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:40.725639] end - ✅ in 0.092s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:41.725983] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:41.824460] end - ✅ in 0.098s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:42.824835] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:42.924969] end - ✅ in 0.100s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:43.925479] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:43.933915] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:44.934251] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:45.024757] end - ✅ in 0.090s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:46.025158] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:46.033707] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:47.034014] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:47.046862] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:48.047431] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:48.056536] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:49.057016] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:49.066463] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:50.066899] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:50.124042] end - ✅ in 0.057s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:51.124479] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:51.133933] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:52.134463] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:52.144703] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:53.145049] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:53.153681] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:54.154160] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:54.162422] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:55.162772] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:55.170885] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:56.171215] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:56.179600] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:57.180041] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:57.188391] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:58.188705] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:58.197657] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:49:59.198136] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:49:59.206509] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:00.207002] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:00.215884] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:01.216285] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:01.224453] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:02.224808] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:02.234318] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:03.234584] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:03.245324] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:04.245611] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:04.254732] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:05.255199] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:05.263685] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:06.264103] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:06.272452] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:07.272772] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:07.282451] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:08.282743] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:08.290969] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:09.291500] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:09.299543] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:10.299860] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:10.307991] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:11.308521] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:11.316845] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:12.317306] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:12.326110] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:13.326648] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:13.336504] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:14.336784] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:14.349341] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:15.349741] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:15.358203] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:16.358620] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:16.366798] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:17.367162] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:17.375656] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:18.375907] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:18.384593] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:19.384913] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:19.395853] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:20.396173] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:20.404835] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:21.405175] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:21.413623] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:22.414001] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:22.422657] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:23.423046] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:23.431789] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:24.432106] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:24.440282] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:25.440630] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:25.449616] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:26.450052] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:26.459228] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:27.459663] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:27.467846] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:28.468210] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:28.477075] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:29.477397] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:29.485435] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:30.485782] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:30.494087] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:31.494448] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:31.502856] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:32.503303] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:32.512068] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:33.512438] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:33.520617] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:34.521029] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:34.529384] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:35.529674] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:35.539264] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:36.539583] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:36.548296] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:37.548639] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:37.557669] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:38.558108] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:38.566464] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:39.566781] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:39.575226] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:40.575611] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:40.584193] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:41.584495] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:41.593214] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:42.593515] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:42.601064] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:43.601428] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:43.610085] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:44.610614] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:44.618354] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:45.618686] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:45.627791] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:46.628077] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:46.636198] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:47.636552] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:47.646866] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:48.647287] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:48.656474] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:49.656866] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:49.665642] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:50.665985] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:50.675585] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:51.675889] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:51.683817] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:52.684096] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:52.702265] end - ✅ in 0.018s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:53.702555] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:53.710899] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:54.711210] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:54.719632] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:55.720084] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:55.728367] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:56.728721] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:56.736526] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:57.736854] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:57.745913] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:58.746288] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:58.755601] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:50:59.756096] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:50:59.764388] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:00.764806] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:00.773174] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:01.773465] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:01.787236] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:02.787498] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:02.795663] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:03.796009] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:03.804343] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:04.804645] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:04.813964] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:05.814420] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:05.823058] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:06.823502] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:06.832232] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:07.832548] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:07.841865] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:08.842194] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:08.850013] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:09.850386] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:09.858988] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:10.859405] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:10.867988] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:11.868391] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:11.877015] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:12.877461] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:12.886068] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:13.886458] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:13.894336] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:14.894631] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:14.902604] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:15.902857] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:15.910994] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:16.911462] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:16.919695] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:17.919979] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:17.928308] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:18.928750] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:18.938753] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:19.939023] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:19.947944] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:20.948253] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:20.956573] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:21.956887] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:21.965174] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:22.965506] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:22.978443] end - ✅ in 0.013s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:23.978760] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:23.987146] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:24.987465] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:24.995640] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:25.995930] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:26.003541] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:27.003860] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:27.011975] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:28.012309] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:28.020286] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:29.020744] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:29.029170] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:30.029447] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:30.037390] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:31.037753] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:31.045445] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:32.045706] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:32.124802] end - ✅ in 0.079s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:33.125279] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:33.141364] end - ✅ in 0.016s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:34.141644] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:34.154177] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:35.154469] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:35.162706] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:36.162993] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:36.170776] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:37.171032] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:37.178945] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:38.179248] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:38.187212] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:39.187513] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:39.195871] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:40.196192] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:40.205212] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:41.205627] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:41.213739] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:42.214039] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:42.222388] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:43.222837] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:43.232630] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:44.232880] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:44.240730] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:45.241024] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:45.249785] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:46.250050] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:46.258342] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:47.258633] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:47.266312] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:48.266633] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:48.274413] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:49.274717] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:49.283403] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:50.283832] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:50.291853] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:51.292203] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:51.301883] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:52.302179] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:52.313779] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:53.314102] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:53.322260] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:54.322550] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:54.334246] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:55.334533] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:55.342403] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:56.342653] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:56.351531] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:57.351869] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:57.360706] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:58.361164] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:58.369502] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:51:59.369847] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:51:59.378052] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:00.378389] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:00.387439] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:01.387864] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:01.396466] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:02.396851] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:02.405027] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:03.405336] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:03.413380] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:04.413632] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:04.421206] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:05.421457] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:05.429648] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:06.429979] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:06.438587] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:07.438872] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:07.447553] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:08.447836] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:08.456528] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:09.456821] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:09.467352] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:10.467607] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:10.492503] end - ✅ in 0.025s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:11.492765] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:11.500539] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:12.500796] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:12.508755] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:13.509046] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:13.517230] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:14.517510] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:14.525263] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:15.525566] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:15.534043] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:16.534365] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:16.543589] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:17.543937] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:17.552538] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:18.552909] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:18.561014] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:19.561408] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:19.569874] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:20.570153] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:20.578168] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:21.578491] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:21.586636] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:22.586916] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:22.595113] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:23.595424] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:23.603206] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:24.603525] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:24.612747] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:25.613196] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:25.636689] end - ✅ in 0.023s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:26.637148] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:26.835828] end - ✅ in 0.198s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:27.836249] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:27.844155] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:28.844450] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:28.852154] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:29.852420] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:29.859516] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:30.859797] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:30.867651] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:31.867955] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:31.875776] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:32.876053] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:32.884339] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:33.884649] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:33.892633] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:34.893027] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:34.903393] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:35.903670] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:35.912226] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:36.912553] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:36.920489] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:37.920785] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:37.929661] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:38.929944] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:38.941008] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:39.941368] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:39.949863] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:40.950378] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:41.035738] end - ✅ in 0.085s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:42.036049] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:42.045680] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:43.045941] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:43.053861] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:44.054193] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:44.062588] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:45.062887] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:45.071368] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:46.071834] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:46.081440] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:47.081849] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:47.090663] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:48.090916] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:48.110355] end - ✅ in 0.019s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:49.110835] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:49.119459] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:50.119878] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:50.127860] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:51.128186] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:51.136844] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:52.137154] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:52.146542] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:53.146984] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:53.155213] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:54.155524] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:54.163768] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:55.164185] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:55.236511] end - ✅ in 0.072s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:56.236852] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:56.335433] end - ✅ in 0.098s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:57.335789] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:57.536061] end - ✅ in 0.200s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:58.536396] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:58.545060] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:52:59.545439] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:52:59.553881] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:00.554332] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:00.562960] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:01.563398] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:01.571592] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:02.571895] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:02.587475] end - ✅ in 0.015s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:03.587791] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:03.635581] end - ✅ in 0.047s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:04.635874] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:04.644354] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:05.644611] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:05.652649] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:06.652972] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:06.661074] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:07.661572] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:07.670256] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:08.670711] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:08.679296] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:09.679668] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:09.688175] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:10.688634] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:10.697514] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:11.697792] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:11.706194] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:12.706665] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:12.715641] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:13.715945] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:13.725100] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:14.725457] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:14.733911] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:15.734204] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:15.742606] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:16.742920] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:16.751741] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:17.752059] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:17.760490] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:18.760884] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:18.768900] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:19.769168] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:19.777393] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:20.777896] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:20.786673] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:21.787048] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:21.795815] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:22.799213] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:22.809294] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:23.809740] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:23.818053] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:24.818377] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:24.826705] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:25.827163] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:25.835615] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:26.836055] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:26.844369] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:27.844642] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:27.852754] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:28.853251] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:28.861671] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:29.862173] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:29.870338] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:30.870590] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:30.878927] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:31.879323] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:31.887863] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:32.888347] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:32.898896] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:33.899380] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:33.908109] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:34.908556] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:34.916951] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:35.917417] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:35.926044] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:36.926451] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:36.936394] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:37.936640] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:37.945218] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:38.945608] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:38.957269] end - ✅ in 0.011s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:39.957566] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:39.965995] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:40.966332] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:40.974608] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:41.974887] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:41.983475] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:42.983782] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:42.992500] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:43.992822] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:44.001361] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:45.001782] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:45.011531] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:46.011900] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:46.020735] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:47.021002] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:47.031234] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:48.031701] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:48.040196] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:49.040512] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:49.049484] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:50.049930] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:50.058080] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:51.058394] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:51.066830] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:52.067090] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:52.075621] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:53.075929] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:53.084653] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:54.085028] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:54.093372] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:55.093684] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:55.101638] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:56.102085] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:56.110023] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:57.110324] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:57.118823] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:58.119274] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:58.128110] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:53:59.128402] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:53:59.137005] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:00.137364] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:00.145692] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:01.145977] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:01.154222] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:02.154502] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:02.162355] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:03.162642] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:03.171052] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:04.171466] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:04.180400] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:05.180764] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:05.188591] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:06.188936] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:06.197417] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:07.197705] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:07.210310] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:08.210565] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:08.219088] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:09.219590] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:09.228746] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:10.229073] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:10.237518] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:11.237793] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:11.246319] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:12.246700] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:12.254701] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:13.254996] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:13.263587] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:14.263843] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:14.271702] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:15.271968] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:15.280276] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:16.280733] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:16.289630] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:17.289959] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:17.298222] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:18.298615] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:18.306933] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:19.307377] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:19.315874] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:20.316139] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:20.324219] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:21.324517] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:21.333008] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:22.333299] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:22.342402] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:23.342824] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:23.352070] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:24.352379] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:24.360428] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:25.360744] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:25.369173] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:26.369464] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:26.379874] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:27.380276] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:27.388729] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:28.389051] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:28.397871] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:29.398283] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:29.406825] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:30.407183] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:30.415893] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:31.416338] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:31.424325] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:32.424990] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:32.433553] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:33.433864] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:33.442212] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:34.442555] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:34.450799] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:35.451308] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:35.459288] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:36.459569] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:36.467939] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:37.468243] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:37.476677] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:38.477004] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:38.489494] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:39.489749] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:39.498302] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:40.498682] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:40.507101] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:41.507632] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:41.516108] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:42.516663] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:42.525099] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:43.525435] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:43.533820] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:44.534194] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:44.543337] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:45.544013] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:45.552500] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:46.552871] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:46.561982] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:47.562318] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:47.569863] end - ✅ in 0.007s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:48.570179] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:48.582879] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:49.583196] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:49.591407] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:50.591859] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:50.600918] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:51.601224] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:51.609470] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:52.609968] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:52.618569] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:53.618979] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:53.627701] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:54.628091] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:54.636982] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:55.637298] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:55.646102] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:56.646414] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:56.654873] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:57.655291] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:57.663522] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:58.663819] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:58.672637] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:54:59.673089] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:54:59.681786] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:00.682349] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:00.690453] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:01.690735] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:01.698775] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:02.699200] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:02.707542] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:03.707965] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:03.718459] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:04.718740] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:04.729396] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:05.729690] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:05.739379] end - ✅ in 0.009s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:06.739655] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:06.747703] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:07.748001] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:07.756354] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:08.756670] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:08.765199] end - ✅ in 0.008s [e2e-llm-inference-service] INFO e2e.llmisvc.test_llm_inference_service:test_llm_inference_service.py:632 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:09.765625] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test', 'v1alpha1'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:09.776369] end - ✅ in 0.011s [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [wait_for_llm_isvc_ready] [2026-04-24T21:55:09.776564] end - ❌ 900.952s: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llmisvc] [2026-04-24T21:55:09.776675] start - args=(, 'custom-route-timeout-pd-test', 'kserve-ci-e2e-test'), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llmisvc] [2026-04-24T21:55:09.788496] end - ✅ in 0.012s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [delete_llmisvc] [2026-04-24T21:55:11.241043] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'custom-route-timeout-pd-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-custom-route-timeout-pd-8b408578'}, [e2e-llm-inference-service] {'name': 'scheduler-managed-custom-route-35ef63b2'}, [e2e-llm-inference-service] {'name': 'workload-pd-cpu-custom-route-ti-2766b49d'}, [e2e-llm-inference-service] {'name': 'model-fb-opt-125m-custom-route-e92a9ea6'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [delete_llmisvc] [2026-04-24T21:55:11.283458] end - ✅ in 0.042s [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [test_llm_inference_service] [2026-04-24T21:55:11.283611] end - ❌ 902.545s: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-04-24T21:42:07Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:28Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-04-24T21:40:58Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-04-24T21:40:42Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] =============================== warnings summary =============================== [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:151 [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:151 [e2e-llm-inference-service] /workspace/source/test/e2e/llmisvc/test_llm_inference_service.py:151: PytestUnknownMarkWarning: Unknown pytest.mark.custom_gateway - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:200 [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:200 [e2e-llm-inference-service] /workspace/source/test/e2e/llmisvc/test_llm_inference_service.py:200: PytestUnknownMarkWarning: Unknown pytest.mark.custom_gateway - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:252 [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:252 [e2e-llm-inference-service] /workspace/source/test/e2e/llmisvc/test_llm_inference_service.py:252: PytestUnknownMarkWarning: Unknown pytest.mark.custom_gateway - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:299 [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:299 [e2e-llm-inference-service] /workspace/source/test/e2e/llmisvc/test_llm_inference_service.py:299: PytestUnknownMarkWarning: Unknown pytest.mark.cluster_gpu - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html [e2e-llm-inference-service] pytest.mark.cluster_gpu, [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:316 [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:316 [e2e-llm-inference-service] /workspace/source/test/e2e/llmisvc/test_llm_inference_service.py:316: PytestUnknownMarkWarning: Unknown pytest.mark.no_scheduler - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html [e2e-llm-inference-service] pytest.mark.no_scheduler, [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:329 [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:329 [e2e-llm-inference-service] /workspace/source/test/e2e/llmisvc/test_llm_inference_service.py:329: PytestUnknownMarkWarning: Unknown pytest.mark.cluster_multi_node - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_multi_node], [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] [e2e-llm-inference-service] /workspace/source/python/kserve/.venv/lib64/python3.11/site-packages/pytest_asyncio/plugin.py:761: DeprecationWarning: The event_loop fixture provided by pytest-asyncio has been redefined in [e2e-llm-inference-service] /workspace/source/test/e2e/conftest.py:43 [e2e-llm-inference-service] Replacing the event_loop fixture with a custom implementation is deprecated [e2e-llm-inference-service] and will lead to errors in the future. [e2e-llm-inference-service] If you want to request an asyncio event loop with a scope other than function [e2e-llm-inference-service] scope, use the "scope" argument to the asyncio mark when marking the tests. [e2e-llm-inference-service] If you want to return different types of event loops, use the event_loop_policy [e2e-llm-inference-service] fixture. [e2e-llm-inference-service] [e2e-llm-inference-service] warnings.warn( [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py:40: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:124: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html [e2e-llm-inference-service] =========================== short test summary info ============================ [e2e-llm-inference-service] FAILED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] FAILED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] ====== 2 failed, 20 passed, 3 skipped, 28 warnings in 2029.99s (0:33:49) ======= [must-gather] [must-gather ] OUT 2026-04-24T22:00:31.304282476Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 16dc56fd-55e9-4d08-86d8-6623a3d5ebfb [must-gather] ClientVersion: 4.21.5 [must-gather] ClusterVersion: Stable at "4.20.19" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-04-24T22:00:31.355331215Z namespace/openshift-must-gather-kjgcf created [must-gather] [must-gather ] OUT 2026-04-24T22:00:31.362899494Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-sq7dh created [must-gather] [must-gather ] OUT 2026-04-24T22:00:31.397890663Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:36.556417698Z [disk usage checker] Started [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:36.589648191Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:36.828333822Z Error from server (NotFound): namespaces "redhat-ods-operator" not found [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:36.991378811Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:36.991410133Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:36.991410133Z namespaces "redhat-ods-operator" not found [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:36.994277587Z Error getting logs from redhat-ods-operator [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.158538492Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.158571320Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.158571320Z namespaces "redhat-ods-monitoring" not found [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.161254682Z Error getting logs from redhat-ods-monitoring [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.326621739Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.326653139Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.326653139Z namespaces "redhat-ods-applications" not found [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.329651647Z Error getting logs from redhat-ods-applications [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.492413666Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.492528612Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.492528612Z namespaces "rhods-notebooks" not found [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.495235224Z Error getting logs from rhods-notebooks [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.658545824Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.658577644Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.658577644Z namespaces "rhoai-model-registries" not found [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.661342649Z Error getting logs from rhoai-model-registries [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.821183340Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.821214874Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.821214874Z namespaces "istio-system" not found [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.824056621Z Error getting logs from istio-system [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.984434466Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.984464010Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.984464010Z namespaces "knative-serving" not found [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:37.987111630Z Error getting logs from knative-serving [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:38.150070977Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:38.150100008Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:38.150100008Z namespaces "redhat-ods-applications-auth-provider" not found [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:38.152860869Z Error getting logs from redhat-ods-applications-auth-provider [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:38.438139181Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:38.719634452Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:38.864409084Z error: the server doesn't have a resource type "auths" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:38.944911585Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:38.947204614Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.079080558Z error: the server doesn't have a resource type "monitorings" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.161914508Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.164474684Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.297661976Z error: the server doesn't have a resource type "featuretrackers" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.382379067Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.385004229Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.518706916Z error: the server doesn't have a resource type "codeflares" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.601645984Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.604285791Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.735678015Z error: the server doesn't have a resource type "dashboards" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.818050994Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.820494267Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:39.956863222Z error: the server doesn't have a resource type "datasciencepipelines" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.039989069Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.042573813Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.176233502Z error: the server doesn't have a resource type "feastoperators" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.256277173Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.258717018Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.396552617Z error: the server doesn't have a resource type "kserves" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.478629504Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.481159496Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.611141331Z error: the server doesn't have a resource type "kueues" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.695022925Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.697517574Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.830359212Z error: the server doesn't have a resource type "modelcontrollers" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.910372840Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:40.912783070Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.043962878Z error: the server doesn't have a resource type "modelmeshservings" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.125318284Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.127758921Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.260215623Z error: the server doesn't have a resource type "modelregistries" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.344317136Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.346984552Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.478620695Z error: the server doesn't have a resource type "rays" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.559411008Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.561941670Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.594710699Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.693770775Z error: the server doesn't have a resource type "trainingoperators" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.775215681Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.778044029Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.910908846Z error: the server doesn't have a resource type "trustyais" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.990928060Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:41.993367330Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:42.120126085Z error: the server doesn't have a resource type "workbenches" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:42.203159597Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:42.205720056Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:42.339465086Z error: the server doesn't have a resource type "hardwareprofiles" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:42.423812444Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:42.426352759Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:42.558592072Z error: the server doesn't have a resource type "llamastackoperators" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:42.636338216Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:42.639105463Z Error collecting info from [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:43.345661280Z error: the server doesn't have a resource type "predictors" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:43.476078447Z error: the server doesn't have a resource type "localmodelnodegroups" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:43.608608997Z error: the server doesn't have a resource type "smcp" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:43.738855498Z error: the server doesn't have a resource type "smm" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:43.866688519Z error: the server doesn't have a resource type "smmr" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:44.458585871Z error: the server doesn't have a resource type "knativeservings" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:44.587408944Z error: the server doesn't have a resource type "configurations" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:44.721642255Z error: the server doesn't have a resource type "routes" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:44.859241098Z error: the server doesn't have a resource type "services" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:44.992473414Z error: the server doesn't have a resource type "revisions" [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:45.522297228Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:45.900539618Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.269296786Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.476509703Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.476553178Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.476553178Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.476553178Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.476553178Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.476553178Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.476553178Z [previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found, container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.481024360Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.599704209Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.638140435Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:46.934753769Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:47.328030076Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:47.536855690Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:47.536890167Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:47.536890167Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:47.536890167Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:47.536890167Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:47.536890167Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:47.536890167Z [previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found, container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:47.540980032Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:47.698773362Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.003985099Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.373496488Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.569272300Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.569316315Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.569316315Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.569316315Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.569316315Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.569316315Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.569316315Z [previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found, container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.573219671Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:48.735947354Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.043475228Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.408362326Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.622309418Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.622352508Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.622352508Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.622352508Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.622352508Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.622352508Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.622352508Z [container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing, previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.626572571Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:49.786936810Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.095410766Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.461340795Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.674236705Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.674283052Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.674283052Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.674283052Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.674283052Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.674283052Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.674283052Z [previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found, container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.678152508Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:50.837763694Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.144073396Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.513860715Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.604185713Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.717133347Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.717176574Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.717176574Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.717176574Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.717176574Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.717176574Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.717176574Z [container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing, previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.721867648Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:51.881791927Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.180274637Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.555434031Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.761542276Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.761746529Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.761746529Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.761746529Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.761746529Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.761746529Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.761746529Z [previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found, container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.766370328Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:52.923147512Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.221005641Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.595617854Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.800237545Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.800285295Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.800285295Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.800285295Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.800285295Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.800285295Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.800285295Z [previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found, container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.804539099Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:53.974044847Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:54.291830818Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:54.668840780Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:54.868618781Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:54.868662700Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:54.868662700Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:54.868662700Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:54.868662700Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:54.868662700Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:54.868662700Z [container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing, previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:54.873006995Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.035547083Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.342540872Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.739057202Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.965294077Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.965331194Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.965331194Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.965331194Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.965331194Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.965331194Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.965331194Z [previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found, container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:55.969032530Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:56.134890288Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:56.366852362Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:56.608564493Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:56.865947882Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.074678506Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.074720488Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.074720488Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.074720488Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.074720488Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.074720488Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.074720488Z [container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing, previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.079470006Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.236706381Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.456271375Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:57.811694554Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.020935580Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.020983782Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.020983782Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.020983782Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.020983782Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.020983782Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.020983782Z [container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing, previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.025268296Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.183867111Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.398354155Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.750183456Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.957590047Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.957635351Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.957635351Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.957635351Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.957635351Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.957635351Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.957635351Z [container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing, previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:58.961726873Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.116729218Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.336363442Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.690940084Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.897034695Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.897076162Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.897076162Z one or more errors occurred while gathering pod-specific data for namespace: kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.897076162Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.897076162Z one or more errors occurred while gathering container data for pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1: [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.897076162Z [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.897076162Z [container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" is waiting to start: PodInitializing, previous terminated container "main" in pod "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" not found] [must-gather] [must-gather-trfp2] POD 2026-04-24T22:00:59.901103561Z Error inspecting namespace/kserve-ci-e2e-test [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:00.066644237Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:00.322628038Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:00.477001503Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:00.705521650Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:00.868114593Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:01.123857599Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:01.293625668Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:01.527734629Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:01.616788886Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:01.693319033Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:02.009482158Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:02.193200114Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:02.460455650Z Wrote inspect data to must-gather. [must-gather] [must-gather-trfp2] POD 2026-04-24T22:01:02.508858845Z Caches written to disk [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.517811857Z waiting for gather to complete [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.521239358Z downloading gather output [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.782774756Z receiving incremental file list [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.797112965Z ./ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.797230788Z aggregated-discovery-api.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.797505055Z aggregated-discovery-apis.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.798231652Z event-filter.html [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.799616055Z timestamp [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.799756358Z version [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.803773543Z cluster-scoped-resources/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.803789084Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.803794014Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.803854355Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.803962358Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.803974238Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804008769Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804076041Z namespaces/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804085221Z namespaces/kserve-ci-e2e-test/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804135332Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804245905Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804301876Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804356447Z namespaces/kserve-ci-e2e-test/apps/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804405238Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.80448582Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804672255Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804848659Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.80490336Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.804983102Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.805043403Z namespaces/kserve-ci-e2e-test/batch/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.805094725Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.805227268Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.805289929Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.805348751Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.805451853Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.805504414Z namespaces/kserve-ci-e2e-test/core/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.805548855Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.805670868Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.805775481Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.808359722Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.808438534Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.80869925Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.808822263Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809027608Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.8091006Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809182071Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809272584Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809322065Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809340405Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809396636Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809504029Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809570591Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809631712Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809694463Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809753925Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809823787Z namespaces/kserve-ci-e2e-test/pods/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809832437Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.809880448Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810017331Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/main/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810025431Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/main/main/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810029071Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/main/main/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810077933Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/main/main/logs/current.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810211926Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/main/main/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810310628Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/main/main/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.81038615Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/main/main/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810436181Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/storage-initializer/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810447731Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/storage-initializer/storage-initializer/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810451821Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/storage-initializer/storage-initializer/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810501953Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/storage-initializer/storage-initializer/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810622876Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810722218Z namespaces/kserve-ci-e2e-test/pods/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1/storage-initializer/storage-initializer/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810771649Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-6c59fbf55c-cm2xx/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810845481Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-6c59fbf55c-cm2xx/router-gateway-1-openshift-default-6c59fbf55c-cm2xx.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.810993294Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-6c59fbf55c-cm2xx/istio-proxy/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811005305Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-6c59fbf55c-cm2xx/istio-proxy/istio-proxy/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811009835Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-6c59fbf55c-cm2xx/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811054616Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-6c59fbf55c-cm2xx/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811276211Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-6c59fbf55c-cm2xx/istio-proxy/istio-proxy/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811372323Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-6c59fbf55c-cm2xx/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811430875Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-6866b85949-pjzzg/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811497026Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-6866b85949-pjzzg/router-gateway-2-openshift-default-6866b85949-pjzzg.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811624449Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-6866b85949-pjzzg/istio-proxy/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.81163284Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-6866b85949-pjzzg/istio-proxy/istio-proxy/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.81163953Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-6866b85949-pjzzg/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811685291Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-6866b85949-pjzzg/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811856925Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-6866b85949-pjzzg/istio-proxy/istio-proxy/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.811963257Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-6866b85949-pjzzg/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812037369Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.81208565Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812359357Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/llm-d-routing-sidecar/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812373817Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/llm-d-routing-sidecar/llm-d-routing-sidecar/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812380437Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812436708Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812600632Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812674994Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812697705Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/main/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812707975Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/main/main/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812712195Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/main/main/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.812749496Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/main/main/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813118565Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/main/main/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813228507Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/main/main/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813260178Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/storage-initializer/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813271558Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/storage-initializer/storage-initializer/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813275508Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/storage-initializer/storage-initializer/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813308859Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/storage-initializer/storage-initializer/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813427972Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813493694Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-5bfbbd4669-ff7n8/storage-initializer/storage-initializer/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813536255Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813578046Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813703478Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/main/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813710689Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/main/main/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813722379Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/main/main/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813744619Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/main/main/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.813945524Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/main/main/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814017326Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/main/main/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814035697Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/storage-initializer/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814051687Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/storage-initializer/storage-initializer/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814062957Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/storage-initializer/storage-initializer/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814096998Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/storage-initializer/storage-initializer/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814245611Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814310043Z namespaces/kserve-ci-e2e-test/pods/router-with-refs-pd-test-kserve-prefill-b694fbd9b-m86qq/storage-initializer/storage-initializer/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814355584Z namespaces/kserve-ci-e2e-test/policy/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814400735Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814450776Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814484227Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814527208Z namespaces/kuadrant-system/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814573759Z namespaces/kuadrant-system/kuadrant-system.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814635461Z namespaces/kuadrant-system/apps.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814675042Z namespaces/kuadrant-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814720893Z namespaces/kuadrant-system/apps/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814765084Z namespaces/kuadrant-system/apps/daemonsets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.814845406Z namespaces/kuadrant-system/apps/deployments.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.815209764Z namespaces/kuadrant-system/apps/replicasets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.815529452Z namespaces/kuadrant-system/apps/statefulsets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.815579413Z namespaces/kuadrant-system/autoscaling/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.815617644Z namespaces/kuadrant-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.815687676Z namespaces/kuadrant-system/batch/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.815749097Z namespaces/kuadrant-system/batch/cronjobs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.81586717Z namespaces/kuadrant-system/batch/jobs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.815925501Z namespaces/kuadrant-system/build.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.815980203Z namespaces/kuadrant-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.816052444Z namespaces/kuadrant-system/build.openshift.io/builds.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.816101555Z namespaces/kuadrant-system/core/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.816192208Z namespaces/kuadrant-system/core/configmaps.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.816377172Z namespaces/kuadrant-system/core/endpoints.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.816496965Z namespaces/kuadrant-system/core/events.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.816967536Z namespaces/kuadrant-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.817050388Z namespaces/kuadrant-system/core/pods.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.817418387Z namespaces/kuadrant-system/core/replicationcontrollers.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.81755755Z namespaces/kuadrant-system/core/secrets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.817808966Z namespaces/kuadrant-system/core/services.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.817933429Z namespaces/kuadrant-system/discovery.k8s.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.81797761Z namespaces/kuadrant-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818087382Z namespaces/kuadrant-system/image.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818106453Z namespaces/kuadrant-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818205105Z namespaces/kuadrant-system/k8s.ovn.org/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818249556Z namespaces/kuadrant-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818355319Z namespaces/kuadrant-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.81842271Z namespaces/kuadrant-system/monitoring.coreos.com/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818486332Z namespaces/kuadrant-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818556913Z namespaces/kuadrant-system/networking.k8s.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818611705Z namespaces/kuadrant-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818685447Z namespaces/kuadrant-system/pods/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818693387Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818742858Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino-68bd676465-p8zxf.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.81884884Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818857481Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/authorino/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818861701Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/authorino/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.818906432Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/authorino/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.819724211Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/authorino/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.819817513Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/authorino/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.819845994Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.819900295Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/authorino-operator-7587b89b76-dnrb2.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.819994138Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820002088Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/manager/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820005718Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/manager/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820022898Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/manager/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820252914Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820322755Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/manager/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820352086Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820385517Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/dns-operator-controller-manager-844548ff4c-2kll4.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820481529Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820491589Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/manager/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820497379Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/manager/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.82052508Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/manager/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820632563Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820715295Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/manager/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820734705Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820764276Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin-6c886788f8-c22xt.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820863688Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820882429Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/kuadrant-console-plugin/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820888749Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/kuadrant-console-plugin/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.820905329Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/kuadrant-console-plugin/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.821004042Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.821056573Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.821085033Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.821142285Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/kuadrant-operator-controller-manager-6ddf9554fc-cshcc.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.821273418Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.821283838Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/manager/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.821305309Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/manager/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.821323339Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/manager/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.822688082Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.822781554Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/manager/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.822830635Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.822898196Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador-limitador-67566c68b4-59664.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823012069Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823020149Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/limitador/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823023729Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/limitador/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823076081Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/limitador/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823215524Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/limitador/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823287926Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/limitador/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823331557Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823385948Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/limitador-operator-controller-manager-c7fb4c8d5-9lv9j.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823502061Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823508991Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/manager/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823512591Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/manager/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823557692Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/manager/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823713276Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823807758Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/manager/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823864279Z namespaces/kuadrant-system/policy/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823923151Z namespaces/kuadrant-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.823978692Z namespaces/kuadrant-system/route.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.824019783Z namespaces/kuadrant-system/route.openshift.io/routes.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.824058994Z namespaces/openshift-ingress/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.824108355Z namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.824245548Z namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.82430547Z namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.824372381Z namespaces/openshift-ingress/apps/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.824423543Z namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.824542355Z namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.824834172Z namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.82514675Z namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.825212681Z namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.825267172Z namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.825351934Z namespaces/openshift-ingress/batch/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.825381215Z namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.825544509Z namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.82560212Z namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.825668392Z namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.825917568Z namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.825934358Z namespaces/openshift-ingress/core/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.825939519Z namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.826387129Z namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.826475021Z namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.826701826Z namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.826777348Z namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827019754Z namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827177098Z namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827371493Z namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827520966Z namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827561317Z namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827635129Z namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.82767868Z namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827747911Z namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827763102Z namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827849814Z namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827896325Z namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827939556Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.827998207Z namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.828029808Z namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.828076589Z namespaces/openshift-ingress/pods/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.828083749Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.828140681Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/istiod-openshift-gateway-55ff986f96-rgd49.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.828321525Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.828333665Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/discovery/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.828338785Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/discovery/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.828359896Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/discovery/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830340003Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830357343Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/discovery/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830366524Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830377904Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830509917Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830519987Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/istio-proxy/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830525017Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830533548Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830721832Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/istio-proxy/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830779123Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830813234Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830864295Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router-default-5fdf56dbd-s82zg.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830976878Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830985588Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/router/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.830991048Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/router/logs/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.831026409Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/router/logs/current.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.831164082Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/router/logs/previous.insecure.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.831246384Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/router/logs/previous.log [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.831280875Z namespaces/openshift-ingress/policy/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.831323976Z namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.831368367Z namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.831416228Z namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.838050635Z [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.838066256Z sent 3,626 bytes received 397,949 bytes 803,150.00 bytes/sec [must-gather] [must-gather-trfp2] OUT 2026-04-24T22:01:03.838070866Z total size is 4,851,004 speedup is 12.08 [must-gather] [must-gather ] OUT 2026-04-24T22:01:03.975626006Z namespace/openshift-must-gather-kjgcf deleted [must-gather] [must-gather] [must-gather] Reprinting Cluster State: [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 16dc56fd-55e9-4d08-86d8-6623a3d5ebfb [must-gather] ClientVersion: 4.21.5 [must-gather] ClusterVersion: Stable at "4.20.19" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-04-24T22:01:04.093409817Z Using must-gather plug-in image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dd19538052f0deb24938546fa0d17135330102ff01b5699610d2fdf82785eaab [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 16dc56fd-55e9-4d08-86d8-6623a3d5ebfb [must-gather] ClientVersion: 4.21.5 [must-gather] ClusterVersion: Stable at "4.20.19" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-04-24T22:01:04.114392154Z namespace/openshift-must-gather-xhf46 created [must-gather] [must-gather ] OUT 2026-04-24T22:01:04.121941073Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-tzgl5 created [must-gather] [must-gather ] OUT 2026-04-24T22:01:04.147199401Z pod for plug-in image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dd19538052f0deb24938546fa0d17135330102ff01b5699610d2fdf82785eaab created [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:05.549496977Z [disk usage checker] Started [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:05.581632436Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:05.771393663Z Gathering data for ns/openshift-cluster-version... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:05.912660824Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:06.002256851Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:06.096445900Z Gathering data for ns/default... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:06.426504376Z Gathering data for ns/openshift... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:06.889811921Z Gathering data for ns/kube-system... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:07.640593459Z Gathering data for ns/openshift-etcd... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:07.986565516Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:07.986599258Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:07.986599258Z namespaces "assisted-installer" not found [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.100633132Z Waiting on subprocesses to finish execution. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.107762509Z INFO: Gathering on-disk MachineConfig from degraded nodes [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.111707870Z INFO: Gathering machine config daemon's old logs from all nodes [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.114894983Z INFO: Gathering HAProxy config files [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.115083934Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.190655963Z INFO: Waiting for node performance related collection to complete ... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.220071237Z INFO: Collecting host service logs for crio [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.220550884Z INFO: Collecting host service logs for kubelet [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.220874433Z INFO: Collecting host service logs for rpm-ostreed [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.221201662Z INFO: Collecting host service logs for ostree-finalize-staged [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.221514766Z INFO: Collecting host service logs for machine-config-daemon-firstboot [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.221822087Z INFO: Collecting host service logs for machine-config-daemon-host [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.222152014Z INFO: Collecting host service logs for NetworkManager [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.222465046Z INFO: Collecting host service logs for openvswitch [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.222753260Z INFO: Collecting host service logs for ovs-configuration [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.223070512Z INFO: Collecting host service logs for ovsdb-server [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.223378485Z INFO: Collecting host service logs for ovs-vswitchd [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.223739350Z INFO: Waiting for worker host service log collection to complete ... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.635104337Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-xfzqb [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.635167879Z insights-runtime-extractor-2mqlt [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.635190916Z insights-runtime-extractor-5dm7q [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.635210776Z insights-runtime-extractor-qbgk9 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:08.677881485Z No resources found in openshift-etcd namespace. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:09.152573196Z INFO: Waiting for on-disk MachineConfig collection to complete ... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:09.152649073Z INFO: on-disk MachineConfig config collection complete. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:09.286164373Z INFO: namespace openshift-frr-k8s not detected. Skipping. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:09.347132760Z INFO: Found 1 replicas - prometheus-k8s-0 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:09.511566934Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:09.557892467Z No resources found [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:09.565786082Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:09.600436548Z error: the server doesn't have a resource type "performanceprofile" [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:09.651894888Z INFO: "sriov-network-operator" not detected. Skipping. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:09.782151299Z INFO: "metallb-operator" not detected. Skipping. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.000746921Z INFO: Worker host service log collection to complete. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.053403283Z error: the server doesn't have a resource type "multi-networkpolicy" [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.183802444Z No resources found [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.251861799Z INFO: 'previous-logs' folder not found on ip-10-0-128-142.ec2.internal, skipping... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.261520274Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.261520274Z See 'oc rsync -h' for help and examples [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.375160732Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.422672583Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.464677816Z INFO: Getting alertmanagers from prometheus-k8s-0 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.511740565Z ERROR: No running kube-apiserver pods found [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.539210640Z INFO: Waiting for HAProxy config collection to complete ... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.665613457Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.670592268Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.909281708Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.926583899Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:10.987014634Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.019721258Z INFO: 'previous-logs' folder not found on ip-10-0-131-162.ec2.internal, skipping... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.056429047Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.088103015Z INFO: HAProxy config collection complete. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.219328782Z INFO: Getting rules from prometheus-k8s-0 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.317325551Z error: the server doesn't have a resource type "machineconfigs" [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.317855285Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.416403116Z INFO: 'previous-logs' folder not found on ip-10-0-139-15.ec2.internal, skipping... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.416525810Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.416584947Z INFO: Machine Config Daemon termination log collection complete. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.504723133Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.648671631Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.729693765Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.785106113Z INFO: Getting status/config from prometheus-k8s-0 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:11.828294262Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.026687456Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.069400700Z Gathering data for ns/openshift-monitoring... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.114869975Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.240626488Z error: the server doesn't have a resource type "kubeletconfigs" [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.319861581Z INFO: Getting status/flags from prometheus-k8s-0 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.486505923Z INFO: INTERCONNECT MODE [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.486707915Z INFO: Gathering ovn-kubernetes DBs [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.556912484Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.583912222Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.705709859Z INFO: Gathering OVN_Northbound from ovnkube-node-6rbsp... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.706007207Z INFO: Gathering OVN_Northbound from ovnkube-node-nxnnq... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.706248403Z INFO: Gathering OVN_Northbound from ovnkube-node-qnlsv... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.706564959Z INFO: Gathering OVN_Southbound from ovnkube-node-6rbsp... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.706836948Z INFO: Gathering OVN_Southbound from ovnkube-node-nxnnq... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.707125009Z INFO: Gathering OVN_Southbound from ovnkube-node-qnlsv... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:12.909583931Z INFO: Getting targets?state=active from prometheus-k8s-0 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.015496813Z tar: Removing leading `/' from member names [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.028880886Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.054408657Z tar: Removing leading `/' from member names [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.066019115Z tar: Removing leading `/' from member names [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.081874978Z tar: Removing leading `/' from member names [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.146785215Z tar: Removing leading `/' from member names [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.179092851Z tar: Removing leading `/' from member names [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.283659937Z INFO: Getting status/tsdb from prometheus-k8s-0 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.422762961Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.750473756Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.807431527Z INFO: Getting status from alertmanager-main-0 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.828537165Z INFO: Waiting for network log collection to complete ... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.828647050Z INFO: Waiting for ovnk database copies to complete ... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.829103344Z INFO: Copying ovnk databases complete. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.830743395Z 5.0M must-gather/network_logs/ovnk_database_store [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.833662574Z ovnk_database_store/ [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.833662574Z ovnk_database_store/ovnkube-node-qnlsv_sbdb [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.868240410Z ovnk_database_store/ovnkube-node-nxnnq_sbdb [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.897526851Z ovnk_database_store/ovnkube-node-6rbsp_nbdb [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.908378483Z ovnk_database_store/ovnkube-node-nxnnq_nbdb [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.920130265Z ovnk_database_store/ovnkube-node-qnlsv_nbdb [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.933950677Z ovnk_database_store/ovnkube-node-6rbsp_sbdb [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:13.972814466Z INFO: Network log collection complete. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:14.067849721Z W0424 22:01:14.067805 1385 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:14.075791141Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:14.373617984Z Gathering data for ns/openshift-network-console... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:14.438190959Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:14.827609372Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:14.884231788Z Gathering data for ns/openshift-console-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:15.126619585Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:15.129071343Z INFO: Fallback to identify the container image from release info [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:15.373339527Z Gathering data for ns/openshift-console... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:15.672204384Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:15.701781685Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:15.750991880Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:15.846061128Z daemonset.apps/perf-node-gather-daemonset created [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:15.891727592Z Gathering data for ns/openshift-cluster-storage-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:15.985501693Z Waiting for performance profile collector pods to become ready: 1 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:16.292771817Z Gathering data for ns/openshift-dns-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:16.627567739Z Gathering data for ns/openshift-dns... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:17.129756218Z Waiting for performance profile collector pods to become ready: 2 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:17.252597071Z Gathering data for ns/openshift-image-registry... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:17.721458898Z Gathering data for ns/openshift-ingress-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:18.049007696Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:18.267252936Z Waiting for performance profile collector pods to become ready: 3 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:18.573446706Z Gathering data for ns/openshift-ingress-canary... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:18.984273694Z Gathering data for ns/openshift-insights... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:19.394508392Z Waiting for performance profile collector pods to become ready: 4 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:20.526190652Z Waiting for performance profile collector pods to become ready: 5 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:20.676802199Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:21.646537029Z Gathering data for ns/openshift-lws-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:21.658240481Z Waiting for performance profile collector pods to become ready: 6 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:22.179739627Z Gathering data for ns/kserve... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:22.796728633Z Waiting for performance profile collector pods to become ready: 7 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:23.913405019Z Daemonset perf-node-gather-daemonset ready 3 out of 3 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:24.045305551Z Gathering data for ns/openshift-config... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:24.182664119Z Collecting performance related data for node ip-10-0-139-15.ec2.internal [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:24.189369311Z Collecting performance related data for node ip-10-0-131-162.ec2.internal [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:24.198194922Z Collecting performance related data for node ip-10-0-128-142.ec2.internal [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:24.396553594Z Gathering data for ns/openshift-config-managed... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:24.945078230Z Gathering data for ns/openshift-kube-apiserver-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:25.316499257Z Gathering data for ns/openshift-kube-apiserver... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:25.686568827Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:25.702728939Z Gathering data for ns/openshift-kube-controller-manager... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:26.057192560Z Gathering data for ns/openshift-kube-controller-manager-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:26.388604185Z Gathering data for ns/openshift-kube-scheduler... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:26.718852607Z Gathering data for ns/openshift-kube-scheduler-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:27.077777396Z Gathering data for ns/openshift-kube-storage-version-migrator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:27.464034748Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:27.777305203Z Gathering data for ns/openshift-user-workload-monitoring... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:28.406210282Z Gathering data for ns/openshift-multus... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:29.807388634Z Gathering data for ns/openshift-ovn-kubernetes... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:30.692135173Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:31.506424497Z Gathering data for ns/openshift-host-network... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:31.858166740Z Gathering data for ns/openshift-network-diagnostics... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:32.472376058Z Gathering data for ns/openshift-network-node-identity... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:32.635562200Z Collecting kubelet logs for node ip-10-0-139-15.ec2.internal [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:32.639089741Z Collecting kubelet logs for node ip-10-0-131-162.ec2.internal [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:32.643301276Z Collecting kubelet logs for node ip-10-0-128-142.ec2.internal [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:32.992103683Z Gathering data for ns/openshift-network-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:33.083181003Z daemonset.apps "perf-node-gather-daemonset" deleted [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:33.089074765Z INFO: Node performance data collection complete. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:33.496389127Z Gathering data for ns/openshift-cloud-network-config-controller... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:33.790447974Z Gathering data for ns/openshift-cluster-node-tuning-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:34.251824171Z Gathering data for ns/openshift-apiserver-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:34.585802092Z Gathering data for ns/openshift-apiserver... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:35.009695401Z Gathering data for ns/openshift-controller-manager-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:35.283677453Z Gathering data for ns/openshift-controller-manager... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:35.573546487Z Gathering data for ns/openshift-cluster-samples-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:35.698727822Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:36.234442946Z Gathering data for ns/openshift-operator-lifecycle-manager... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:36.532491021Z Gathering data for ns/openshift-service-ca-operator... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:36.938560948Z Gathering data for ns/openshift-service-ca... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:37.303199216Z Gathering data for ns/openshift-cluster-csi-drivers... [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:38.224877916Z Wrote inspect data to must-gather. [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:38.224937527Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:38.224937527Z skipping gathering secrets/support due to error: secrets "support" not found [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:38.363016567Z error: the server doesn't have a resource type "clusters" [must-gather] [must-gather-5jw29] POD 2026-04-24T22:01:38.615532844Z Caches written to disk [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.623061364Z waiting for gather to complete [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.626481165Z downloading gather output [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.898367446Z receiving incremental file list [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.918017952Z ./ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.918103874Z aggregated-discovery-api.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.918423921Z aggregated-discovery-apis.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.919111168Z event-filter.html [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.921706549Z timestamp [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.921796581Z version [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.93019943Z cluster-scoped-resources/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.930217061Z cluster-scoped-resources/admissionregistration.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.930233411Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.930264032Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/default-network-annotation.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.930385605Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/servicecidrs.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.930478227Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/user-defined-networks-namespace-label.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.930547508Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.93059184Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/default-network-annotation-binding.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.930690552Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/servicecidrs-binding.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.930774364Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/user-defined-networks-namespace-label-binding.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.930864986Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.930903897Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/multus.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.93102376Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/network-node-identity.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.931147253Z cluster-scoped-resources/apiextensions.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.931491041Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.931539592Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/accounts.nim.opendatahub.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.931752297Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/adminnetworkpolicies.policy.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.932005383Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/adminpolicybasedexternalroutes.k8s.ovn.org.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.932246219Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertingrules.monitoring.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.932393752Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertmanagerconfigs.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.935831624Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertmanagers.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.937394071Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertrelabelconfigs.monitoring.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.937514553Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apikeys.devportal.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.937657197Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apiproducts.devportal.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.938145539Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apirequestcounts.apiserver.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.938305372Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apiservers.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.938458346Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/appliedmanifestworks.work.open-cluster-management.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.938591039Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authconfigs.authorino.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.939388958Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authentications.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.939627084Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorinos.operator.authorino.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.939751786Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorizationpolicies.security.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.939916001Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authpolicies.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.941087478Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/baselineadminnetworkpolicies.policy.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.941347474Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/builds.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.941535889Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/bundles.trust.cert-manager.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.941696743Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/catalogsources.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.941982959Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certificaterequests.cert-manager.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.942140223Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certificates.cert-manager.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.942350708Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certmanagers.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.942626235Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/challenges.acme.cert-manager.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.943322581Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudcredentials.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.943438824Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudeventsources.eventing.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.943545346Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudprivateipconfigs.cloud.network.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.943666759Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterclaims.cluster.open-cluster-management.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.943773672Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustercloudeventsources.eventing.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.943883694Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustercsidrivers.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.944046328Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterimagepolicies.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.944239423Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterissuers.cert-manager.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.945039202Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusteroperators.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.945193775Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterresourcequotas.quota.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.94536269Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterserviceversions.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.947212073Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterstoragecontainers.serving.kserve.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.947378087Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustertriggerauthentications.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.947531461Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusteruserdefinednetworks.k8s.ovn.org.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.947751746Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterversions.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.948032183Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.948484273Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.948600946Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.samples.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.948723569Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleclidownloads.console.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.948833372Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleexternalloglinks.console.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.948966845Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolelinks.console.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.949079418Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolenotifications.console.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.949221321Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleplugins.console.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.949393975Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolequickstarts.console.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.949532798Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoles.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.949648291Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoles.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.949942748Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolesamples.console.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.950077821Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleyamlsamples.console.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.950215265Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/controllerconfigs.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.950750357Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/credentialsrequests.cloudcredential.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.95088285Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/csisnapshotcontrollers.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.950986183Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/datascienceclusters.datasciencecluster.opendatahub.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.951226658Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/destinationrules.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.951939765Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnses.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.952057098Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnses.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.952282374Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnshealthcheckprobes.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.952399566Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnspolicies.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.95255725Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnsrecords.ingress.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.952675193Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnsrecords.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.952838247Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dscinitializations.dscinitialization.opendatahub.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.95298832Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressfirewalls.k8s.ovn.org.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.953109483Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressips.k8s.ovn.org.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.953250246Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressqoses.k8s.ovn.org.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.95338824Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressrouters.network.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.953539563Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressservices.k8s.ovn.org.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.953646046Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/envoyfilters.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.953792459Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/etcds.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.953939913Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/featuregates.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.954054605Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gatewayclasses.gateway.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.95422896Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.gateway.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.954601108Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.954789453Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/grpcroutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.955203903Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/helmchartrepositories.helm.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.955323546Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/httproutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.956106454Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagecontentpolicies.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.956250057Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagecontentsourcepolicies.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.95635871Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagedigestmirrorsets.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.956477843Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagepolicies.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.956648057Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagepruners.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.956909023Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/images.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.957039626Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagetagmirrorsets.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.95719488Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencegraphs.serving.kserve.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.957363234Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencemodelrewrites.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.957482377Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferenceobjectives.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.9576067Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepoolimports.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.957745813Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.957902227Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.95803232Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferenceservices.serving.kserve.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.960216762Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/infrastructures.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.960656602Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ingresscontrollers.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.961280007Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ingresses.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.961476391Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/insightsoperators.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.961630255Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/installplans.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.961787888Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ipamclaims.k8s.cni.cncf.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.961910642Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ippools.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.962020554Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/issuers.cert-manager.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.962841293Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/istiocnis.sailoperator.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.963192962Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/istiocsrs.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.963502019Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/istiorevisions.sailoperator.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.968835126Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/istiorevisiontags.sailoperator.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.968983919Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/istios.sailoperator.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.970514705Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kedacontrollers.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.97200209Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kuadrants.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.972180475Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubeapiservers.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.972360009Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubecontrollermanagers.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.972511933Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubeschedulers.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.972658386Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubestorageversionmigrators.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.97283743Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/leaderworkersetoperators.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.972989104Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/leaderworkersets.leaderworkerset.x-k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.980741188Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/limitadors.limitador.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.981054695Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/llminferenceserviceconfigs.serving.kserve.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.985557052Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/llminferenceservices.serving.kserve.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.98969737Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineconfignodes.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.989852893Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineconfigpools.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.990053208Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineconfigurations.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.990368656Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineosbuilds.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.990515909Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineosconfigs.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.990666163Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/network-attachment-definitions.k8s.cni.cncf.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.990751715Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/networks.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.990914909Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/networks.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.991201075Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/nodes.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.991305218Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/nodeslicepools.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.99141448Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/oauths.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.991627435Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/oidcpolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.991757258Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/olmconfigs.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.991876491Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/openshiftapiservers.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.991982834Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/openshiftcontrollermanagers.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.992107287Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorconditions.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.992269761Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorgroups.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.992409624Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorhubs.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.992522387Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorpkis.network.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.992635239Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operators.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.992755522Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/orders.acme.cert-manager.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.992896576Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/overlappingrangeipreservations.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.993014428Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/peerauthentications.security.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.993172492Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/pinnedimagesets.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.993290455Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/planpolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.993415008Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/podmonitors.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.993661534Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/podnetworkconnectivitychecks.controlplane.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.993793717Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/probes.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.994020892Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/profiles.tuned.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.994177726Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/projecthelmchartrepositories.helm.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.994302349Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/projects.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.994405311Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/prometheuses.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.996302976Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/prometheusrules.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.996424159Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxies.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.996533582Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxyconfigs.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.996647425Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/rangeallocations.security.internal.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.996755247Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ratelimitpolicies.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.996907381Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/referencegrants.gateway.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.997044474Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/requestauthentications.security.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.997226438Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/rolebindingrestrictions.authorization.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.997352871Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/scaledjobs.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.998829636Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/scaledobjects.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.99898931Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/schedulers.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.999103673Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/securitycontextconstraints.security.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.999283057Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servicecas.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.99940239Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/serviceentries.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.999584014Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servicemonitors.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:39.9998287Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servingruntimes.serving.kserve.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.000375793Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/sidecars.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.000643009Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storages.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.000763512Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storagestates.migration.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.000876235Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storageversionmigrations.migration.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.001009268Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/subscriptions.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.001622762Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetries.telemetry.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.001827207Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetrypolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.00194823Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/thanosrulers.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.003337873Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tlspolicies.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.003505877Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tokenratelimitpolicies.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.003656251Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/trainedmodels.serving.kserve.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.003763873Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/triggerauthentications.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.003914306Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/trustmanagers.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.004210984Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tuneds.tuned.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.004348337Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/userdefinednetworks.k8s.ovn.org.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.00449487Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/virtualservices.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.00488172Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumepopulators.populator.storage.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.004998292Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshotclasses.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.005117665Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshotcontents.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.00532422Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshots.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.005475634Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/wasmplugins.extensions.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.005612697Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadentries.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.00575252Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadgroups.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.005926334Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ztunnels.sailoperator.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.00701365Z cluster-scoped-resources/apiregistration.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.007226415Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.007287646Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1..yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.007387159Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.acme.cert-manager.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.007474231Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.admissionregistration.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.007563733Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apiextensions.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.007641175Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apiserver.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.007816349Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apps.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.007924502Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008014284Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authentication.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008107156Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authorization.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008218829Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authorization.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008313381Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.autoscaling.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008403933Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.batch.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008493535Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.build.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008583377Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cert-manager.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008671449Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.certificates.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008767732Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cloud.network.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008854264Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cloudcredential.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.008965356Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.config.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.009054828Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.console.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.009164011Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.coordination.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.009262543Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.datasciencecluster.opendatahub.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.009353355Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.discovery.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.009438497Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.dscinitialization.opendatahub.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.00953069Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.events.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.009625082Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.flowcontrol.apiserver.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.009711274Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.gateway.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.009804726Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.image.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.009899268Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.009988581Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.inference.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.010078063Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.ingress.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.010192675Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.k8s.cni.cncf.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.010280348Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.k8s.ovn.org.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.01036449Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.010464622Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.leaderworkerset.x-k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.010552944Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.010645266Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.010735468Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.monitoring.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.01082726Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.network.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.010921303Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.011010985Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.011102427Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.nim.opendatahub.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.011212779Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.node.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.011307662Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.oauth.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.011403344Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.011492796Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.011589328Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.packages.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.011682741Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.policy.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.011801793Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.project.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.011905836Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.quota.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.012006208Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.rbac.authorization.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.01208867Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.route.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.012212413Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.sailoperator.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.012306835Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.samples.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.012407468Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.scheduling.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.0124982Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.internal.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.012596192Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.012691065Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.012785797Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.012881559Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.storage.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.012989141Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.telemetry.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.013077914Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.template.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.013223267Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.tuned.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.013325989Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.user.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.013424382Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.work.open-cluster-management.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.013544305Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.cluster.open-cluster-management.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.013644567Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.controlplane.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.013743379Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.devportal.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.013839742Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.eventing.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.013935354Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.extensions.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.014029406Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.extensions.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.014138239Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.014247421Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.k8s.cni.cncf.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.014342824Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.014435636Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.014527498Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.limitador.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.01463182Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.migration.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.014709712Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.014828615Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.operator.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.014913437Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.015005199Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.policy.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.015095342Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.sailoperator.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.015219805Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.serving.kserve.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.015312147Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.telemetry.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.015403919Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.trust.cert-manager.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.015499511Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.015605834Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.015685885Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.015788098Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.serving.kserve.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.01587925Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha3.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.015972522Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.admissionregistration.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.016066434Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.external.metrics.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.016225488Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.gateway.networking.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.016328471Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.helm.openshift.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.016425743Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.016527045Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.metrics.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.016618938Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.monitoring.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.01671499Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.networking.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.016808482Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.operator.authorino.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.016905614Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.populator.storage.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017011277Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.security.istio.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017099589Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.serving.kserve.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017225282Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta2.authorino.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017315044Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta3.authorino.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017413797Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v2.autoscaling.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017499728Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v2.operators.coreos.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.01755401Z cluster-scoped-resources/certificates.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017602481Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017650402Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-2pd52.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017753594Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-2pqcg.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017849677Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-6jx5b.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.017954099Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-6zw8j.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.018024641Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-c6hrj.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.018117243Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-crtg4.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.018224936Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-dvmpx.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.018317198Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-h7mj2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.018485752Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-lrcbz.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.018578024Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-r5v8r.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.018705697Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-zpk2d.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.018782679Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-zqg58.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.018884581Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/system:openshift:openshift-monitoring-bcqxt.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.018990994Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/system:openshift:openshift-monitoring-gc7fl.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.019094726Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/system:openshift:openshift-monitoring-pgt8s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.01924649Z cluster-scoped-resources/config.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.019288521Z cluster-scoped-resources/config.openshift.io/apiservers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.019456895Z cluster-scoped-resources/config.openshift.io/authentications.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.019615178Z cluster-scoped-resources/config.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.019714711Z cluster-scoped-resources/config.openshift.io/clusterimagepolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.019806153Z cluster-scoped-resources/config.openshift.io/clusteroperators.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.020138561Z cluster-scoped-resources/config.openshift.io/clusterversions.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.020235853Z cluster-scoped-resources/config.openshift.io/consoles.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.020332966Z cluster-scoped-resources/config.openshift.io/dnses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.020423328Z cluster-scoped-resources/config.openshift.io/featuregates.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02054614Z cluster-scoped-resources/config.openshift.io/imagecontentpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.020635673Z cluster-scoped-resources/config.openshift.io/imagedigestmirrorsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.020726815Z cluster-scoped-resources/config.openshift.io/imagepolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.020834197Z cluster-scoped-resources/config.openshift.io/images.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02093425Z cluster-scoped-resources/config.openshift.io/imagetagmirrorsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.021022502Z cluster-scoped-resources/config.openshift.io/infrastructures.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.021138775Z cluster-scoped-resources/config.openshift.io/ingresses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.021266748Z cluster-scoped-resources/config.openshift.io/networks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02136584Z cluster-scoped-resources/config.openshift.io/nodes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.021448612Z cluster-scoped-resources/config.openshift.io/oauths.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.021536284Z cluster-scoped-resources/config.openshift.io/operatorhubs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.021607926Z cluster-scoped-resources/config.openshift.io/projects.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.021698398Z cluster-scoped-resources/config.openshift.io/proxies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02178254Z cluster-scoped-resources/config.openshift.io/schedulers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.021874092Z cluster-scoped-resources/config.openshift.io/clusteroperators/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.021930963Z cluster-scoped-resources/config.openshift.io/clusteroperators/console.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.022030196Z cluster-scoped-resources/config.openshift.io/clusteroperators/csi-snapshot-controller.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.022146118Z cluster-scoped-resources/config.openshift.io/clusteroperators/dns.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.022249541Z cluster-scoped-resources/config.openshift.io/clusteroperators/image-registry.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.022339733Z cluster-scoped-resources/config.openshift.io/clusteroperators/ingress.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.022436865Z cluster-scoped-resources/config.openshift.io/clusteroperators/insights.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.022532278Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-apiserver.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.022676521Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-controller-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.022775244Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-scheduler.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.022860895Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-storage-version-migrator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.022954368Z cluster-scoped-resources/config.openshift.io/clusteroperators/monitoring.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02304895Z cluster-scoped-resources/config.openshift.io/clusteroperators/network.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.023205674Z cluster-scoped-resources/config.openshift.io/clusteroperators/node-tuning.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.023305456Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-apiserver.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.023400798Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-controller-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02348914Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-samples.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.023592453Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager-catalog.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.023677955Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager-packageserver.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.023773457Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.023863169Z cluster-scoped-resources/config.openshift.io/clusteroperators/service-ca.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.023961812Z cluster-scoped-resources/config.openshift.io/clusteroperators/storage.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024038453Z cluster-scoped-resources/config.openshift.io/clusterversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024087275Z cluster-scoped-resources/config.openshift.io/clusterversions/version.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024200037Z cluster-scoped-resources/config.openshift.io/consoles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024242298Z cluster-scoped-resources/config.openshift.io/consoles/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02430648Z cluster-scoped-resources/config.openshift.io/featuregates/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024339901Z cluster-scoped-resources/config.openshift.io/featuregates/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024427952Z cluster-scoped-resources/config.openshift.io/infrastructures/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024467004Z cluster-scoped-resources/config.openshift.io/infrastructures/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024532655Z cluster-scoped-resources/config.openshift.io/oauths/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024566206Z cluster-scoped-resources/config.openshift.io/oauths/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024624287Z cluster-scoped-resources/config.openshift.io/proxies/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024672258Z cluster-scoped-resources/config.openshift.io/proxies/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02473351Z cluster-scoped-resources/console.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02474701Z cluster-scoped-resources/console.openshift.io/consoleplugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024809102Z cluster-scoped-resources/console.openshift.io/consoleplugins/kuadrant-console-plugin.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.024906634Z cluster-scoped-resources/console.openshift.io/consoleplugins/monitoring-plugin.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.025010166Z cluster-scoped-resources/console.openshift.io/consoleplugins/networking-console-plugin.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.025057707Z cluster-scoped-resources/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.025070748Z cluster-scoped-resources/core/nodes/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.025112759Z cluster-scoped-resources/core/nodes/ip-10-0-128-142.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.025351325Z cluster-scoped-resources/core/nodes/ip-10-0-131-162.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.025496708Z cluster-scoped-resources/core/nodes/ip-10-0-139-15.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02560072Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.025673722Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.025704973Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/catch-all.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.025813685Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/endpoint-controller.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.025908778Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/exempt.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.0259901Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/global-default.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.026071851Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-controller-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.026180914Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-scheduler.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.026335518Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-system-service-accounts.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02642868Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.026543503Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver-sar.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.026614664Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.026710167Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-authentication-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.026807359Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-controller-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.026904671Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-kube-apiserver-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.027000613Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-monitoring-metrics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.027094726Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-apiserver-sar.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02727345Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-apiserver.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02729118Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-server.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.027385943Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/probes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.027572867Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/service-accounts.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.027583527Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-leader-election.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.027657599Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-node-high.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.027745271Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-nodes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.027839553Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/workload-leader-election.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.027927986Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.027959776Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/catch-all.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028053078Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/exempt.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028153231Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/global-default.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028253173Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/leader-election.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028333645Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/node-high.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028413157Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/openshift-control-plane-operators.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028569861Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/system.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028689154Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/workload-high.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028788896Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/workload-low.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028844087Z cluster-scoped-resources/gateway.networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028854808Z cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028896249Z cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/openshift-default.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.028992941Z cluster-scoped-resources/imageregistry.operator.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029005451Z cluster-scoped-resources/imageregistry.operator.openshift.io/configs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029037952Z cluster-scoped-resources/imageregistry.operator.openshift.io/configs/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029186235Z cluster-scoped-resources/imageregistry.operator.openshift.io/imagepruners/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029209756Z cluster-scoped-resources/imageregistry.operator.openshift.io/imagepruners/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029288008Z cluster-scoped-resources/migration.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029297268Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029344449Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/console-plugin-storage-version-migration.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029445871Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/machineconfiguration-controllerconfig-storage-version-migration.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029597405Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/machineconfiguration-machineconfigpool-storage-version-migration.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029648896Z cluster-scoped-resources/oauth.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029662607Z cluster-scoped-resources/oauth.openshift.io/oauthclients/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029694687Z cluster-scoped-resources/oauth.openshift.io/oauthclients/console.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.02982046Z cluster-scoped-resources/operator.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029832921Z cluster-scoped-resources/operator.openshift.io/clustercsidrivers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029842121Z cluster-scoped-resources/operator.openshift.io/clustercsidrivers/ebs.csi.aws.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029940873Z cluster-scoped-resources/operator.openshift.io/consoles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.029993884Z cluster-scoped-resources/operator.openshift.io/consoles/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030106767Z cluster-scoped-resources/operator.openshift.io/csisnapshotcontrollers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030205029Z cluster-scoped-resources/operator.openshift.io/csisnapshotcontrollers/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030340303Z cluster-scoped-resources/operator.openshift.io/dnses/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030395554Z cluster-scoped-resources/operator.openshift.io/dnses/default.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030466055Z cluster-scoped-resources/operator.openshift.io/insightsoperators/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030502896Z cluster-scoped-resources/operator.openshift.io/insightsoperators/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03064383Z cluster-scoped-resources/operator.openshift.io/kubeapiservers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030693291Z cluster-scoped-resources/operator.openshift.io/kubeapiservers/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030756573Z cluster-scoped-resources/operator.openshift.io/kubecontrollermanagers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030774523Z cluster-scoped-resources/operator.openshift.io/kubecontrollermanagers/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030858815Z cluster-scoped-resources/operator.openshift.io/kubeschedulers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030899806Z cluster-scoped-resources/operator.openshift.io/kubeschedulers/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030942867Z cluster-scoped-resources/operator.openshift.io/kubestorageversionmigrators/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.030992908Z cluster-scoped-resources/operator.openshift.io/kubestorageversionmigrators/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031042159Z cluster-scoped-resources/operator.openshift.io/networks/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03108485Z cluster-scoped-resources/operator.openshift.io/networks/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031194633Z cluster-scoped-resources/operator.openshift.io/openshiftapiservers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031245084Z cluster-scoped-resources/operator.openshift.io/openshiftapiservers/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031310496Z cluster-scoped-resources/operator.openshift.io/openshiftcontrollermanagers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031347596Z cluster-scoped-resources/operator.openshift.io/openshiftcontrollermanagers/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031414268Z cluster-scoped-resources/operator.openshift.io/servicecas/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031437579Z cluster-scoped-resources/operator.openshift.io/servicecas/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031524431Z cluster-scoped-resources/operator.openshift.io/storages/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031560921Z cluster-scoped-resources/operator.openshift.io/storages/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031665454Z cluster-scoped-resources/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031673834Z cluster-scoped-resources/operators.coreos.com/olmconfigs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031714165Z cluster-scoped-resources/operators.coreos.com/olmconfigs/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031787887Z cluster-scoped-resources/operators.coreos.com/operators/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.031823418Z cluster-scoped-resources/operators.coreos.com/operators/authorino-operator.kuadrant-system.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03193445Z cluster-scoped-resources/operators.coreos.com/operators/dns-operator.kuadrant-system.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.032032053Z cluster-scoped-resources/operators.coreos.com/operators/leader-worker-set.openshift-lws-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.032227477Z cluster-scoped-resources/operators.coreos.com/operators/limitador-operator.kuadrant-system.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03233917Z cluster-scoped-resources/operators.coreos.com/operators/openshift-cert-manager-operator.cert-manager-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.032450033Z cluster-scoped-resources/operators.coreos.com/operators/openshift-custom-metrics-autoscaler-operator.openshift-keda.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.032552445Z cluster-scoped-resources/operators.coreos.com/operators/rhcl-operator.kuadrant-system.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.032660878Z cluster-scoped-resources/operators.coreos.com/operators/servicemeshoperator3.openshift-operators.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03274733Z cluster-scoped-resources/rbac.authorization.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03276721Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.032811591Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/aws-ebs-csi-driver-operator-clusterrolebinding.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.032909704Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cloud-network-config-controller.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.032993746Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cluster-storage-operator-role.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03317974Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/metrics-daemon-sa-rolebinding.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.033280952Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-admission-controller-webhook.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.033368344Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-ancillary-tools.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.033461357Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-cluster-readers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.033544908Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-group.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.033630601Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-transient.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.033723333Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-whereabouts.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.033806395Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/network-diagnostics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.033900657Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/network-node-identity.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.033984529Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-image-registry-pruner.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.034081901Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-iptables-alerter.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.034201594Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.034300057Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-node-identity-limited.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.034385179Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-node-kube-rbac-proxy.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03446993Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/registry-registry-role.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.034574643Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.034611854Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/aws-ebs-csi-driver-operator-clusterrole.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.034722307Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/cloud-network-config-controller.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.034800988Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/metrics-daemon-role.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03488734Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus-admission-controller-webhook.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035036764Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus-ancillary-tools.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035134426Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035248029Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/net-attach-def-project.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035331271Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/network-diagnostics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035425923Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/network-node-identity.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035517995Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-iptables-alerter.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035607437Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-cluster-reader.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035691579Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035797132Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-kube-rbac-proxy.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035883004Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-node-limited.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.035964556Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-udn-editor.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036051558Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-udn-viewer.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036172851Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:registry.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036269573Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/whereabouts-cni.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036344105Z cluster-scoped-resources/sailoperator.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036362045Z cluster-scoped-resources/sailoperator.io/istios/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036380126Z cluster-scoped-resources/sailoperator.io/istios/openshift-gateway.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036463178Z cluster-scoped-resources/samples.operator.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036474708Z cluster-scoped-resources/samples.operator.openshift.io/configs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036508039Z cluster-scoped-resources/samples.operator.openshift.io/configs/cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03655921Z cluster-scoped-resources/snapshot.storage.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03656737Z cluster-scoped-resources/snapshot.storage.k8s.io/volumesnapshotclasses/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036611701Z cluster-scoped-resources/snapshot.storage.k8s.io/volumesnapshotclasses/csi-aws-vsc.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036673013Z cluster-scoped-resources/storage.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036680293Z cluster-scoped-resources/storage.k8s.io/csidrivers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036739234Z cluster-scoped-resources/storage.k8s.io/csidrivers/ebs.csi.aws.com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036803186Z cluster-scoped-resources/storage.k8s.io/csinodes/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036842917Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-128-142.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.036954609Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-131-162.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037041371Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-139-15.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037093183Z cluster-scoped-resources/storage.k8s.io/storageclasses/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037148994Z cluster-scoped-resources/storage.k8s.io/storageclasses/gp2-csi.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037256927Z cluster-scoped-resources/storage.k8s.io/storageclasses/gp3-csi.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037288337Z host_service_logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037338989Z host_service_logs/masters/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03738133Z host_service_logs/masters/NetworkManager_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037455341Z host_service_logs/masters/crio_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037527163Z host_service_logs/masters/kubelet_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037586584Z host_service_logs/masters/machine-config-daemon-firstboot_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037710477Z host_service_logs/masters/machine-config-daemon-host_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03782051Z host_service_logs/masters/openvswitch_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037885011Z host_service_logs/masters/ostree-finalize-staged_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.037959473Z host_service_logs/masters/ovs-configuration_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038036685Z host_service_logs/masters/ovs-vswitchd_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038102247Z host_service_logs/masters/ovsdb-server_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038226379Z host_service_logs/masters/rpm-ostreed_service.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03825686Z ingress_controllers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038270111Z ingress_controllers/default/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038273911Z ingress_controllers/default/router-default-5fdf56dbd-s82zg/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038328882Z ingress_controllers/default/router-default-5fdf56dbd-s82zg/haproxy.config [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038479996Z insights-data/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038489216Z machine_config_ondisk/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038493116Z machine_config_termination_logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038499406Z monitoring/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038508606Z monitoring/alertmanager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038535467Z monitoring/alertmanager/status.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038633839Z monitoring/alertmanager/status.stderr [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.03868795Z monitoring/prometheus/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038735452Z monitoring/prometheus/alertmanagers.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038835124Z monitoring/prometheus/alertmanagers.stderr [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.038906756Z monitoring/prometheus/rules.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.039671104Z monitoring/prometheus/rules.stderr [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.039719685Z monitoring/prometheus/prometheus-k8s-0/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.039773256Z monitoring/prometheus/prometheus-k8s-0/active-targets.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.040653407Z monitoring/prometheus/prometheus-k8s-0/active-targets.stderr [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.040704008Z monitoring/prometheus/prometheus-k8s-0/status/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.040723509Z monitoring/prometheus/prometheus-k8s-0/status/runtimeinfo.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.040813991Z monitoring/prometheus/prometheus-k8s-0/status/runtimeinfo.stderr [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.040899133Z monitoring/prometheus/prometheus-k8s-0/status/tsdb.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.041000125Z monitoring/prometheus/prometheus-k8s-0/status/tsdb.stderr [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.041063057Z monitoring/prometheus/status/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.041108818Z monitoring/prometheus/status/config.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.041584419Z monitoring/prometheus/status/config.stderr [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.041654951Z monitoring/prometheus/status/flags.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.041753183Z monitoring/prometheus/status/flags.stderr [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.041945927Z namespaces/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.041956648Z namespaces/cert-manager-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.041961608Z namespaces/cert-manager-operator/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.041965268Z namespaces/cert-manager-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.042003179Z namespaces/cert-manager-operator/coordination.k8s.io/leases/cert-manager-operator-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.04206211Z namespaces/cert-manager-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.042081091Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.042149402Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.042369428Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.042600403Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.042750747Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.042964532Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.043292759Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.043560536Z namespaces/cert-manager-operator/operators.coreos.com/installplans/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.043596887Z namespaces/cert-manager-operator/operators.coreos.com/installplans/install-brx8g.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.043755981Z namespaces/cert-manager-operator/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.043800922Z namespaces/cert-manager-operator/operators.coreos.com/operatorconditions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.043866673Z namespaces/cert-manager-operator/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.043904454Z namespaces/cert-manager-operator/operators.coreos.com/operatorgroups/cert-manager-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.043960265Z namespaces/cert-manager-operator/operators.coreos.com/subscriptions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.044010256Z namespaces/cert-manager-operator/operators.coreos.com/subscriptions/openshift-cert-manager-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.044084578Z namespaces/cert-manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.044094238Z namespaces/cert-manager/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.044102479Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.04417236Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.044351995Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.044536649Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.044684383Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.044881547Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.045145863Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.045402709Z namespaces/default/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.045454741Z namespaces/default/default.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.045524972Z namespaces/default/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.045568373Z namespaces/default/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.045639975Z namespaces/default/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.045679416Z namespaces/default/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.045770588Z namespaces/default/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.04585012Z namespaces/default/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.045935722Z namespaces/default/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.045976073Z namespaces/default/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.046019214Z namespaces/default/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.046073425Z namespaces/default/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.046114226Z namespaces/default/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.046224959Z namespaces/default/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.04628673Z namespaces/default/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.046339252Z namespaces/default/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.046411593Z namespaces/default/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.046499526Z namespaces/default/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.046538196Z namespaces/default/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.04668565Z namespaces/default/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.046778302Z namespaces/default/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.046952566Z namespaces/default/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047040198Z namespaces/default/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047139671Z namespaces/default/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047282494Z namespaces/default/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047446988Z namespaces/default/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.04754419Z namespaces/default/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047590321Z namespaces/default/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047666793Z namespaces/default/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047708754Z namespaces/default/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047764185Z namespaces/default/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047804626Z namespaces/default/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047892479Z namespaces/default/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.047971521Z namespaces/default/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.048024072Z namespaces/default/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.048072813Z namespaces/default/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.048109704Z namespaces/default/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.048201796Z namespaces/default/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.048253657Z namespaces/default/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.048265167Z namespaces/default/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.048298358Z namespaces/default/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.048457762Z namespaces/default/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.048648626Z namespaces/default/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.04879621Z namespaces/default/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049029505Z namespaces/default/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049323462Z namespaces/default/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049509027Z namespaces/default/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049549988Z namespaces/default/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049590119Z namespaces/default/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.04963548Z namespaces/default/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049699031Z namespaces/kserve-ci-e2e-test/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049708142Z namespaces/kserve-ci-e2e-test/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049712902Z namespaces/kserve-ci-e2e-test/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049750483Z namespaces/kserve-ci-e2e-test/coordination.k8s.io/leases/epp-kserve-ci-e2e-test-scheduler-ha-replicas-test-inference-pool.gateway-api-inference-extension.sigs.k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049860905Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049871035Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.049912646Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/router-gateway-1.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.050014449Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/router-gateway-2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.050088471Z namespaces/kserve-ci-e2e-test/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.050100761Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.050206873Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.050370747Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.050566232Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.050699055Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.05089403Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.051145205Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.051402522Z namespaces/kserve/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.051439872Z namespaces/kserve/kserve.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.051509044Z namespaces/kserve/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.051539455Z namespaces/kserve/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.051619787Z namespaces/kserve/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.051648748Z namespaces/kserve/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.05174121Z namespaces/kserve/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.051946295Z namespaces/kserve/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052143879Z namespaces/kserve/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052212371Z namespaces/kserve/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052249582Z namespaces/kserve/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052284803Z namespaces/kserve/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052336644Z namespaces/kserve/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052418606Z namespaces/kserve/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052487897Z namespaces/kserve/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052535918Z namespaces/kserve/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.05261477Z namespaces/kserve/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052661661Z namespaces/kserve/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052670882Z namespaces/kserve/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052706753Z namespaces/kserve/coordination.k8s.io/leases/kserve-controller-manager-leader-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052793775Z namespaces/kserve/coordination.k8s.io/leases/llminferenceservice-kserve-controller-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.052881387Z namespaces/kserve/coordination.k8s.io/leases/odh-model-controller.opendatahub.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.053002869Z namespaces/kserve/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.053044461Z namespaces/kserve/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.053334788Z namespaces/kserve/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.053461691Z namespaces/kserve/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.053756157Z namespaces/kserve/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.053834659Z namespaces/kserve/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.054081895Z namespaces/kserve/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.054234409Z namespaces/kserve/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.054475504Z namespaces/kserve/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.054576357Z namespaces/kserve/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.054620668Z namespaces/kserve/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.05471311Z namespaces/kserve/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.054738851Z namespaces/kserve/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.054825063Z namespaces/kserve/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.054849263Z namespaces/kserve/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.054948666Z namespaces/kserve/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055014697Z namespaces/kserve/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055063588Z namespaces/kserve/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055102179Z namespaces/kserve/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055295554Z namespaces/kserve/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055345785Z namespaces/kserve/monitoring.coreos.com/servicemonitors/model-serving-api-metrics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055431247Z namespaces/kserve/monitoring.coreos.com/servicemonitors/odh-model-controller-metrics-monitor.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055480048Z namespaces/kserve/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055524679Z namespaces/kserve/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055606941Z namespaces/kserve/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055616231Z namespaces/kserve/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055656423Z namespaces/kserve/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055807996Z namespaces/kserve/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.055996911Z namespaces/kserve/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.056142944Z namespaces/kserve/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.056435251Z namespaces/kserve/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.056574384Z namespaces/kserve/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.056861031Z namespaces/kserve/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.056877461Z namespaces/kserve/pods/kserve-controller-manager-74fc8f6f96-bfwqh/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.056909042Z namespaces/kserve/pods/kserve-controller-manager-74fc8f6f96-bfwqh/kserve-controller-manager-74fc8f6f96-bfwqh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057048765Z namespaces/kserve/pods/kserve-controller-manager-74fc8f6f96-bfwqh/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057057806Z namespaces/kserve/pods/kserve-controller-manager-74fc8f6f96-bfwqh/manager/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057061426Z namespaces/kserve/pods/kserve-controller-manager-74fc8f6f96-bfwqh/manager/manager/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057103007Z namespaces/kserve/pods/kserve-controller-manager-74fc8f6f96-bfwqh/manager/manager/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057309512Z namespaces/kserve/pods/kserve-controller-manager-74fc8f6f96-bfwqh/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057341803Z namespaces/kserve/pods/kserve-controller-manager-74fc8f6f96-bfwqh/manager/manager/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057394554Z namespaces/kserve/pods/llmisvc-controller-manager-6f7fd9578-hv8sp/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057426225Z namespaces/kserve/pods/llmisvc-controller-manager-6f7fd9578-hv8sp/llmisvc-controller-manager-6f7fd9578-hv8sp.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057501466Z namespaces/kserve/pods/llmisvc-controller-manager-6f7fd9578-hv8sp/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057532267Z namespaces/kserve/pods/llmisvc-controller-manager-6f7fd9578-hv8sp/manager/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057541687Z namespaces/kserve/pods/llmisvc-controller-manager-6f7fd9578-hv8sp/manager/manager/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.057579298Z namespaces/kserve/pods/llmisvc-controller-manager-6f7fd9578-hv8sp/manager/manager/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059225267Z namespaces/kserve/pods/llmisvc-controller-manager-6f7fd9578-hv8sp/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059296689Z namespaces/kserve/pods/llmisvc-controller-manager-6f7fd9578-hv8sp/manager/manager/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.05935799Z namespaces/kserve/pods/model-serving-api-86f7b4b499-znlgd/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059391981Z namespaces/kserve/pods/model-serving-api-86f7b4b499-znlgd/model-serving-api-86f7b4b499-znlgd.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059493973Z namespaces/kserve/pods/model-serving-api-86f7b4b499-znlgd/server/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059500843Z namespaces/kserve/pods/model-serving-api-86f7b4b499-znlgd/server/server/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059506743Z namespaces/kserve/pods/model-serving-api-86f7b4b499-znlgd/server/server/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059556315Z namespaces/kserve/pods/model-serving-api-86f7b4b499-znlgd/server/server/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059646227Z namespaces/kserve/pods/model-serving-api-86f7b4b499-znlgd/server/server/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059713788Z namespaces/kserve/pods/model-serving-api-86f7b4b499-znlgd/server/server/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.05976916Z namespaces/kserve/pods/odh-model-controller-696fc77849-ft9w6/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059812431Z namespaces/kserve/pods/odh-model-controller-696fc77849-ft9w6/odh-model-controller-696fc77849-ft9w6.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059894693Z namespaces/kserve/pods/odh-model-controller-696fc77849-ft9w6/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059901833Z namespaces/kserve/pods/odh-model-controller-696fc77849-ft9w6/manager/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059916343Z namespaces/kserve/pods/odh-model-controller-696fc77849-ft9w6/manager/manager/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.059960524Z namespaces/kserve/pods/odh-model-controller-696fc77849-ft9w6/manager/manager/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068342993Z namespaces/kserve/pods/odh-model-controller-696fc77849-ft9w6/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068411445Z namespaces/kserve/pods/odh-model-controller-696fc77849-ft9w6/manager/manager/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068437605Z namespaces/kserve/pods/s3-init-8h7vn/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068486126Z namespaces/kserve/pods/s3-init-8h7vn/s3-init-8h7vn.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068574278Z namespaces/kserve/pods/s3-init-8h7vn/s3-init/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068582479Z namespaces/kserve/pods/s3-init-8h7vn/s3-init/s3-init/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068587099Z namespaces/kserve/pods/s3-init-8h7vn/s3-init/s3-init/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068612759Z namespaces/kserve/pods/s3-init-8h7vn/s3-init/s3-init/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068716752Z namespaces/kserve/pods/s3-init-8h7vn/s3-init/s3-init/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068789923Z namespaces/kserve/pods/s3-init-8h7vn/s3-init/s3-init/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068823754Z namespaces/kserve/pods/seaweedfs-86cc847c5c-4xfmc/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068870205Z namespaces/kserve/pods/seaweedfs-86cc847c5c-4xfmc/seaweedfs-86cc847c5c-4xfmc.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068961318Z namespaces/kserve/pods/seaweedfs-86cc847c5c-4xfmc/seaweedfs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068970728Z namespaces/kserve/pods/seaweedfs-86cc847c5c-4xfmc/seaweedfs/seaweedfs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.068976188Z namespaces/kserve/pods/seaweedfs-86cc847c5c-4xfmc/seaweedfs/seaweedfs/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069001408Z namespaces/kserve/pods/seaweedfs-86cc847c5c-4xfmc/seaweedfs/seaweedfs/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069228444Z namespaces/kserve/pods/seaweedfs-86cc847c5c-4xfmc/seaweedfs/seaweedfs/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069301626Z namespaces/kserve/pods/seaweedfs-86cc847c5c-4xfmc/seaweedfs/seaweedfs/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069347177Z namespaces/kserve/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069390718Z namespaces/kserve/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069452859Z namespaces/kserve/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.06950309Z namespaces/kserve/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069588913Z namespaces/kuadrant-system/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069641734Z namespaces/kuadrant-system/kuadrant-system.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069709495Z namespaces/kuadrant-system/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069763326Z namespaces/kuadrant-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069831308Z namespaces/kuadrant-system/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069880789Z namespaces/kuadrant-system/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.069962671Z namespaces/kuadrant-system/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.070269618Z namespaces/kuadrant-system/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.070534485Z namespaces/kuadrant-system/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.070580066Z namespaces/kuadrant-system/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.070633847Z namespaces/kuadrant-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.070692639Z namespaces/kuadrant-system/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.07074287Z namespaces/kuadrant-system/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.070813522Z namespaces/kuadrant-system/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.070894363Z namespaces/kuadrant-system/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.070933274Z namespaces/kuadrant-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071035067Z namespaces/kuadrant-system/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071064537Z namespaces/kuadrant-system/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071070408Z namespaces/kuadrant-system/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071114639Z namespaces/kuadrant-system/coordination.k8s.io/leases/3745a16e.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071234901Z namespaces/kuadrant-system/coordination.k8s.io/leases/a3f98d6c.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071314253Z namespaces/kuadrant-system/coordination.k8s.io/leases/aac3a15d.authorino.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071395785Z namespaces/kuadrant-system/coordination.k8s.io/leases/f139389e.kuadrant.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071476477Z namespaces/kuadrant-system/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071521508Z namespaces/kuadrant-system/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071669532Z namespaces/kuadrant-system/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.071776654Z namespaces/kuadrant-system/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.072193434Z namespaces/kuadrant-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.072270476Z namespaces/kuadrant-system/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.072590494Z namespaces/kuadrant-system/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.072725737Z namespaces/kuadrant-system/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.072976803Z namespaces/kuadrant-system/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073077395Z namespaces/kuadrant-system/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073096336Z namespaces/kuadrant-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073234429Z namespaces/kuadrant-system/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.0732789Z namespaces/kuadrant-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073318261Z namespaces/kuadrant-system/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073358332Z namespaces/kuadrant-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073445414Z namespaces/kuadrant-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073525696Z namespaces/kuadrant-system/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073565597Z namespaces/kuadrant-system/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073605158Z namespaces/kuadrant-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073660879Z namespaces/kuadrant-system/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.07369998Z namespaces/kuadrant-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073760061Z namespaces/kuadrant-system/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073772192Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.073830673Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.074012567Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.074280034Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.074430087Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.074632102Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.074902968Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.075164635Z namespaces/kuadrant-system/operators.coreos.com/installplans/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.075224406Z namespaces/kuadrant-system/operators.coreos.com/installplans/install-8kfls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.075464652Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.075508353Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.075599295Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.075678397Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.075764569Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.07582517Z namespaces/kuadrant-system/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.075872842Z namespaces/kuadrant-system/operators.coreos.com/operatorgroups/kuadrant.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.075927773Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.075970174Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/authorino-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.076077416Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/dns-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.076273301Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/limitador-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.076371733Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/rhcl-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.076446935Z namespaces/kuadrant-system/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.076453925Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.076503016Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino-68bd676465-p8zxf.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.076594688Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.076601359Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/authorino/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.076607179Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/authorino/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.07666084Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/authorino/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.077421738Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/authorino/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.07749079Z namespaces/kuadrant-system/pods/authorino-68bd676465-p8zxf/authorino/authorino/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.077549341Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.077595882Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/authorino-operator-7587b89b76-dnrb2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.077698295Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.077712015Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.077715995Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/manager/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.077723745Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/manager/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.07789865Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.077965801Z namespaces/kuadrant-system/pods/authorino-operator-7587b89b76-dnrb2/manager/manager/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078022872Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078057273Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/dns-operator-controller-manager-844548ff4c-2kll4.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078173916Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078186486Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078200466Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/manager/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078245568Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/manager/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.07835412Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078423442Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-844548ff4c-2kll4/manager/manager/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078480783Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078521574Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin-6c886788f8-c22xt.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078609996Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078620207Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/kuadrant-console-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078628237Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/kuadrant-console-plugin/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078675348Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/kuadrant-console-plugin/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.07874743Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078811791Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-6c886788f8-c22xt/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078862122Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078896493Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/kuadrant-operator-controller-manager-6ddf9554fc-cshcc.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.078997865Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.079010266Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.079014196Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/manager/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.079057317Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/manager/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.08045342Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.080501231Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-6ddf9554fc-cshcc/manager/manager/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.080547472Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.080577243Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador-limitador-67566c68b4-59664.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.080684045Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.080698446Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/limitador/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.080703666Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/limitador/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.080721356Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/limitador/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.080805408Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/limitador/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.08087427Z namespaces/kuadrant-system/pods/limitador-limitador-67566c68b4-59664/limitador/limitador/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.080907111Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.080962282Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/limitador-operator-controller-manager-c7fb4c8d5-9lv9j.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081048004Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081055704Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081067834Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/manager/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081115975Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/manager/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081284319Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081353581Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-c7fb4c8d5-9lv9j/manager/manager/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081391642Z namespaces/kuadrant-system/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081421303Z namespaces/kuadrant-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081479734Z namespaces/kuadrant-system/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081517805Z namespaces/kuadrant-system/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081591357Z namespaces/kube-node-lease/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081612117Z namespaces/kube-node-lease/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081617338Z namespaces/kube-node-lease/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081656878Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-128-142.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.08174385Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-131-162.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081828843Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-139-15.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081892624Z namespaces/kube-node-lease/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081899634Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.081938075Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.082094139Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.082292643Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.082437327Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.082640602Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.082872347Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.083075842Z namespaces/kube-public/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.083083762Z namespaces/kube-public/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.083090512Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.083160074Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.083319888Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.083491462Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.083629305Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.08383368Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.084077416Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.084346712Z namespaces/kube-system/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.084388253Z namespaces/kube-system/kube-system.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.084532187Z namespaces/kube-system/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.084554117Z namespaces/kube-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.084648389Z namespaces/kube-system/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.08468216Z namespaces/kube-system/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.084811663Z namespaces/kube-system/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.084890155Z namespaces/kube-system/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.084986357Z namespaces/kube-system/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085033539Z namespaces/kube-system/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.0850837Z namespaces/kube-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085159502Z namespaces/kube-system/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085205032Z namespaces/kube-system/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085289265Z namespaces/kube-system/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085353386Z namespaces/kube-system/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085405827Z namespaces/kube-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085480699Z namespaces/kube-system/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.08553002Z namespaces/kube-system/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.08553673Z namespaces/kube-system/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085571931Z namespaces/kube-system/coordination.k8s.io/leases/apiserver-wpb7yo2v56eqxn53hddkga5x3q.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085662413Z namespaces/kube-system/coordination.k8s.io/leases/cert-manager-cainjector-leader-election.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085746335Z namespaces/kube-system/coordination.k8s.io/leases/cert-manager-controller.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085830587Z namespaces/kube-system/coordination.k8s.io/leases/kube-controller-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.085990301Z namespaces/kube-system/coordination.k8s.io/leases/kube-scheduler.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.086031702Z namespaces/kube-system/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.086078983Z namespaces/kube-system/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.086304838Z namespaces/kube-system/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.086401861Z namespaces/kube-system/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.086637637Z namespaces/kube-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.086713628Z namespaces/kube-system/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.086976774Z namespaces/kube-system/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.087101468Z namespaces/kube-system/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.087473826Z namespaces/kube-system/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.087533928Z namespaces/kube-system/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.087604499Z namespaces/kube-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.087676821Z namespaces/kube-system/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.087711462Z namespaces/kube-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.087759953Z namespaces/kube-system/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.087805204Z namespaces/kube-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.087888276Z namespaces/kube-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.087964078Z namespaces/kube-system/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.088008049Z namespaces/kube-system/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.088079271Z namespaces/kube-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.088116161Z namespaces/kube-system/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.088186793Z namespaces/kube-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.088245164Z namespaces/kube-system/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.088256575Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.088307026Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.08846226Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.088641034Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.088778027Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.088990682Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089260579Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089469764Z namespaces/kube-system/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089478104Z namespaces/kube-system/pods/global-pull-secret-syncer-9gh4c/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089524365Z namespaces/kube-system/pods/global-pull-secret-syncer-9gh4c/global-pull-secret-syncer-9gh4c.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089611027Z namespaces/kube-system/pods/global-pull-secret-syncer-9gh4c/global-pull-secret-syncer/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089619727Z namespaces/kube-system/pods/global-pull-secret-syncer-9gh4c/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089624347Z namespaces/kube-system/pods/global-pull-secret-syncer-9gh4c/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089668958Z namespaces/kube-system/pods/global-pull-secret-syncer-9gh4c/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089886343Z namespaces/kube-system/pods/global-pull-secret-syncer-9gh4c/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089956845Z namespaces/kube-system/pods/global-pull-secret-syncer-9gh4c/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.089992016Z namespaces/kube-system/pods/global-pull-secret-syncer-lgtm8/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090025877Z namespaces/kube-system/pods/global-pull-secret-syncer-lgtm8/global-pull-secret-syncer-lgtm8.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090103999Z namespaces/kube-system/pods/global-pull-secret-syncer-lgtm8/global-pull-secret-syncer/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090112329Z namespaces/kube-system/pods/global-pull-secret-syncer-lgtm8/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090147329Z namespaces/kube-system/pods/global-pull-secret-syncer-lgtm8/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090199911Z namespaces/kube-system/pods/global-pull-secret-syncer-lgtm8/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090414696Z namespaces/kube-system/pods/global-pull-secret-syncer-lgtm8/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090482528Z namespaces/kube-system/pods/global-pull-secret-syncer-lgtm8/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090516698Z namespaces/kube-system/pods/global-pull-secret-syncer-nbsck/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090543039Z namespaces/kube-system/pods/global-pull-secret-syncer-nbsck/global-pull-secret-syncer-nbsck.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090613871Z namespaces/kube-system/pods/global-pull-secret-syncer-nbsck/global-pull-secret-syncer/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090621311Z namespaces/kube-system/pods/global-pull-secret-syncer-nbsck/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090632601Z namespaces/kube-system/pods/global-pull-secret-syncer-nbsck/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090664462Z namespaces/kube-system/pods/global-pull-secret-syncer-nbsck/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090884077Z namespaces/kube-system/pods/global-pull-secret-syncer-nbsck/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090941828Z namespaces/kube-system/pods/global-pull-secret-syncer-nbsck/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.090980079Z namespaces/kube-system/pods/konnectivity-agent-jf6mk/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.09102654Z namespaces/kube-system/pods/konnectivity-agent-jf6mk/konnectivity-agent-jf6mk.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.091105522Z namespaces/kube-system/pods/konnectivity-agent-jf6mk/konnectivity-agent/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.091115723Z namespaces/kube-system/pods/konnectivity-agent-jf6mk/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.091138643Z namespaces/kube-system/pods/konnectivity-agent-jf6mk/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.091182344Z namespaces/kube-system/pods/konnectivity-agent-jf6mk/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.092156547Z namespaces/kube-system/pods/konnectivity-agent-jf6mk/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.0922648Z namespaces/kube-system/pods/konnectivity-agent-jf6mk/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.09228744Z namespaces/kube-system/pods/konnectivity-agent-kgjxk/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.092343821Z namespaces/kube-system/pods/konnectivity-agent-kgjxk/konnectivity-agent-kgjxk.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.092412063Z namespaces/kube-system/pods/konnectivity-agent-kgjxk/konnectivity-agent/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.092423034Z namespaces/kube-system/pods/konnectivity-agent-kgjxk/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.092436604Z namespaces/kube-system/pods/konnectivity-agent-kgjxk/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.092473655Z namespaces/kube-system/pods/konnectivity-agent-kgjxk/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.093476459Z namespaces/kube-system/pods/konnectivity-agent-kgjxk/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.0935494Z namespaces/kube-system/pods/konnectivity-agent-kgjxk/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.093590611Z namespaces/kube-system/pods/konnectivity-agent-t6vpk/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.093625572Z namespaces/kube-system/pods/konnectivity-agent-t6vpk/konnectivity-agent-t6vpk.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.093716024Z namespaces/kube-system/pods/konnectivity-agent-t6vpk/konnectivity-agent/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.093725224Z namespaces/kube-system/pods/konnectivity-agent-t6vpk/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.093729894Z namespaces/kube-system/pods/konnectivity-agent-t6vpk/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.093773726Z namespaces/kube-system/pods/konnectivity-agent-t6vpk/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.0943765Z namespaces/kube-system/pods/konnectivity-agent-t6vpk/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.094433131Z namespaces/kube-system/pods/konnectivity-agent-t6vpk/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.094474962Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-142.ec2.internal/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.094513623Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-142.ec2.internal/kube-apiserver-proxy-ip-10-0-128-142.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.094588085Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-142.ec2.internal/haproxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.094597135Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-142.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.094606345Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-142.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.094639416Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-142.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.09479237Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-142.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.094869221Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-142.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.094913493Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-131-162.ec2.internal/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.094967544Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-131-162.ec2.internal/kube-apiserver-proxy-ip-10-0-131-162.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095025245Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-131-162.ec2.internal/haproxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095037565Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-131-162.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095043016Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-131-162.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095093317Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-131-162.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095357613Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-131-162.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095423685Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-131-162.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095464806Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-139-15.ec2.internal/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095492926Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-139-15.ec2.internal/kube-apiserver-proxy-ip-10-0-139-15.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095565118Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-139-15.ec2.internal/haproxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095572508Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-139-15.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095578918Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-139-15.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095618289Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-139-15.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095825694Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-139-15.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095899436Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-139-15.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095954057Z namespaces/kube-system/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.095997828Z namespaces/kube-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096046449Z namespaces/kube-system/rbac.authorization.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096054309Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.09609549Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/network-diagnostics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096175212Z namespaces/kube-system/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096215513Z namespaces/kube-system/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096258484Z namespaces/open-cluster-management-29cf724e-1e5c-43f8-ba9b-d885d4b2d/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096266705Z namespaces/open-cluster-management-29cf724e-1e5c-43f8-ba9b-d885d4b2d/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096270285Z namespaces/open-cluster-management-29cf724e-1e5c-43f8-ba9b-d885d4b2d/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096312436Z namespaces/open-cluster-management-29cf724e-1e5c-43f8-ba9b-d885d4b2d/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096467609Z namespaces/open-cluster-management-29cf724e-1e5c-43f8-ba9b-d885d4b2d/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096704435Z namespaces/open-cluster-management-29cf724e-1e5c-43f8-ba9b-d885d4b2d/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.096865089Z namespaces/open-cluster-management-29cf724e-1e5c-43f8-ba9b-d885d4b2d/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.097094414Z namespaces/open-cluster-management-29cf724e-1e5c-43f8-ba9b-d885d4b2d/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.097384321Z namespaces/open-cluster-management-29cf724e-1e5c-43f8-ba9b-d885d4b2d/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.097657478Z namespaces/open-cluster-management-agent-addon/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.097665378Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.097676408Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.097730239Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/cluster-proxy.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.097820441Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/work-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.097868792Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.097878973Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.097936584Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.098095698Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.098299013Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.098428246Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.098639191Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.098888456Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099112002Z namespaces/openshift-apiserver-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099232115Z namespaces/openshift-apiserver-operator/openshift-apiserver-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099290466Z namespaces/openshift-apiserver-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099329277Z namespaces/openshift-apiserver-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099397509Z namespaces/openshift-apiserver-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.09944505Z namespaces/openshift-apiserver-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099523182Z namespaces/openshift-apiserver-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099613884Z namespaces/openshift-apiserver-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099686086Z namespaces/openshift-apiserver-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099742217Z namespaces/openshift-apiserver-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099790668Z namespaces/openshift-apiserver-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.09985415Z namespaces/openshift-apiserver-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.09988897Z namespaces/openshift-apiserver-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.099977242Z namespaces/openshift-apiserver-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.100028183Z namespaces/openshift-apiserver-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.100070685Z namespaces/openshift-apiserver-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.100170427Z namespaces/openshift-apiserver-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.100241319Z namespaces/openshift-apiserver-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.100274489Z namespaces/openshift-apiserver-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.101265153Z namespaces/openshift-apiserver-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.101348295Z namespaces/openshift-apiserver-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.101429707Z namespaces/openshift-apiserver-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.101508989Z namespaces/openshift-apiserver-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.101594341Z namespaces/openshift-apiserver-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.101718804Z namespaces/openshift-apiserver-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.101898548Z namespaces/openshift-apiserver-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.101959219Z namespaces/openshift-apiserver-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.101997Z namespaces/openshift-apiserver-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102054202Z namespaces/openshift-apiserver-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102086612Z namespaces/openshift-apiserver-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102151974Z namespaces/openshift-apiserver-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102204545Z namespaces/openshift-apiserver-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102284097Z namespaces/openshift-apiserver-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102363299Z namespaces/openshift-apiserver-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.10240444Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102451521Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102503832Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102551154Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors/openshift-apiserver-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102605665Z namespaces/openshift-apiserver-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102646546Z namespaces/openshift-apiserver-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102687007Z namespaces/openshift-apiserver-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102696107Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102753478Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.102898172Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.103079706Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.10324323Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.103455745Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.10369755Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.103891485Z namespaces/openshift-apiserver-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.103929436Z namespaces/openshift-apiserver-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.103989757Z namespaces/openshift-apiserver-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.104018628Z namespaces/openshift-apiserver-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.10410774Z namespaces/openshift-apiserver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.104162882Z namespaces/openshift-apiserver/openshift-apiserver.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.104225933Z namespaces/openshift-apiserver/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.104265734Z namespaces/openshift-apiserver/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.104326625Z namespaces/openshift-apiserver/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.104375777Z namespaces/openshift-apiserver/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.104447728Z namespaces/openshift-apiserver/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.10453276Z namespaces/openshift-apiserver/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.104611932Z namespaces/openshift-apiserver/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.104665873Z namespaces/openshift-apiserver/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.104704014Z namespaces/openshift-apiserver/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105015302Z namespaces/openshift-apiserver/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105066433Z namespaces/openshift-apiserver/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105168616Z namespaces/openshift-apiserver/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105210317Z namespaces/openshift-apiserver/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105256227Z namespaces/openshift-apiserver/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.10535092Z namespaces/openshift-apiserver/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105393741Z namespaces/openshift-apiserver/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105430172Z namespaces/openshift-apiserver/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105542504Z namespaces/openshift-apiserver/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105621156Z namespaces/openshift-apiserver/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105701588Z namespaces/openshift-apiserver/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.10577941Z namespaces/openshift-apiserver/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105867662Z namespaces/openshift-apiserver/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.105982775Z namespaces/openshift-apiserver/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106163029Z namespaces/openshift-apiserver/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106221281Z namespaces/openshift-apiserver/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106271692Z namespaces/openshift-apiserver/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106324553Z namespaces/openshift-apiserver/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106376124Z namespaces/openshift-apiserver/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106430475Z namespaces/openshift-apiserver/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106463536Z namespaces/openshift-apiserver/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106554368Z namespaces/openshift-apiserver/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.10662897Z namespaces/openshift-apiserver/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106682751Z namespaces/openshift-apiserver/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106739813Z namespaces/openshift-apiserver/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106795584Z namespaces/openshift-apiserver/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106834425Z namespaces/openshift-apiserver/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106891726Z namespaces/openshift-apiserver/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106906457Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.106948658Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.107137432Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.107397958Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.107537312Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.107737976Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.107985022Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108210528Z namespaces/openshift-apiserver/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108252038Z namespaces/openshift-apiserver/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.10830804Z namespaces/openshift-apiserver/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108346461Z namespaces/openshift-apiserver/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108407812Z namespaces/openshift-authentication-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108415672Z namespaces/openshift-authentication-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108419682Z namespaces/openshift-authentication-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108461103Z namespaces/openshift-authentication-operator/monitoring.coreos.com/servicemonitors/authentication-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108521445Z namespaces/openshift-authentication-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108539575Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108585096Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.10874708Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.108944095Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.109079408Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.109298693Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.109533569Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.109742274Z namespaces/openshift-authentication/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.109756734Z namespaces/openshift-authentication/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.109764424Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.109798445Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.109963799Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.110145253Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.110301747Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.110509312Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.110748648Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.110962683Z namespaces/openshift-cloud-controller-manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.110972143Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.110976713Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.111012854Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/leases/cloud-controller-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.111078065Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.111089506Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.111166358Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.111318631Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.111502875Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.111614318Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.111837603Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112075489Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112318125Z namespaces/openshift-cloud-credential-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112339235Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112344266Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112360806Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/leases/cloud-credential-operator-leader.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112434348Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112449248Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112459628Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/servicemonitors/cloud-credential-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.11253415Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.11254888Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112596801Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112748795Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.112931309Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.113066432Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.113289428Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.113518323Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.113732328Z namespaces/openshift-cloud-network-config-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.113778609Z namespaces/openshift-cloud-network-config-controller/openshift-cloud-network-config-controller.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.113844001Z namespaces/openshift-cloud-network-config-controller/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.113878402Z namespaces/openshift-cloud-network-config-controller/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.113937753Z namespaces/openshift-cloud-network-config-controller/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.113971494Z namespaces/openshift-cloud-network-config-controller/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114065186Z namespaces/openshift-cloud-network-config-controller/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114168249Z namespaces/openshift-cloud-network-config-controller/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.11424602Z namespaces/openshift-cloud-network-config-controller/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114287732Z namespaces/openshift-cloud-network-config-controller/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114321962Z namespaces/openshift-cloud-network-config-controller/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114382644Z namespaces/openshift-cloud-network-config-controller/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114419715Z namespaces/openshift-cloud-network-config-controller/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114509357Z namespaces/openshift-cloud-network-config-controller/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114560108Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114601119Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114679711Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114726912Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114736272Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114782893Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/leases/cloud-network-config-controller-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114853215Z namespaces/openshift-cloud-network-config-controller/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114888846Z namespaces/openshift-cloud-network-config-controller/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.114996178Z namespaces/openshift-cloud-network-config-controller/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.11508075Z namespaces/openshift-cloud-network-config-controller/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115229114Z namespaces/openshift-cloud-network-config-controller/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115311626Z namespaces/openshift-cloud-network-config-controller/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115388958Z namespaces/openshift-cloud-network-config-controller/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115530271Z namespaces/openshift-cloud-network-config-controller/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115636153Z namespaces/openshift-cloud-network-config-controller/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115693605Z namespaces/openshift-cloud-network-config-controller/core/serviceaccounts/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115732296Z namespaces/openshift-cloud-network-config-controller/core/serviceaccounts/cloud-network-config-controller.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115790307Z namespaces/openshift-cloud-network-config-controller/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115848018Z namespaces/openshift-cloud-network-config-controller/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115892279Z namespaces/openshift-cloud-network-config-controller/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.1159329Z namespaces/openshift-cloud-network-config-controller/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.115999172Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116048633Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116152196Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116231427Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116288309Z namespaces/openshift-cloud-network-config-controller/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.11633766Z namespaces/openshift-cloud-network-config-controller/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116373011Z namespaces/openshift-cloud-network-config-controller/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116419882Z namespaces/openshift-cloud-network-config-controller/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116467373Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116496554Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116542235Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116694649Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.116871743Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117019226Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117268092Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117498037Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117699512Z namespaces/openshift-cloud-network-config-controller/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117716193Z namespaces/openshift-cloud-network-config-controller/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117774354Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117787784Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117810765Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/rolebindings/cloud-network-config-controller-rb.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117877357Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117917997Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/roles/cloud-network-config-controller.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.117969849Z namespaces/openshift-cloud-network-config-controller/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118003669Z namespaces/openshift-cloud-network-config-controller/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118078631Z namespaces/openshift-cluster-csi-drivers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118149503Z namespaces/openshift-cluster-csi-drivers/openshift-cluster-csi-drivers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118224495Z namespaces/openshift-cluster-csi-drivers/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118268846Z namespaces/openshift-cluster-csi-drivers/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118317017Z namespaces/openshift-cluster-csi-drivers/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118358608Z namespaces/openshift-cluster-csi-drivers/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118485921Z namespaces/openshift-cluster-csi-drivers/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118561553Z namespaces/openshift-cluster-csi-drivers/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118640775Z namespaces/openshift-cluster-csi-drivers/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118686276Z namespaces/openshift-cluster-csi-drivers/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118730337Z namespaces/openshift-cluster-csi-drivers/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118775568Z namespaces/openshift-cluster-csi-drivers/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118815039Z namespaces/openshift-cluster-csi-drivers/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118896001Z namespaces/openshift-cluster-csi-drivers/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118950672Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.118998623Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.119075605Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.119146547Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.119159817Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.119216408Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/ebs-csi-aws-com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.11931102Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-attacher-leader-ebs-csi-aws-com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.119395433Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-resizer-ebs-csi-aws-com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.119516465Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-snapshotter-leader-ebs-csi-aws-com.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.119610547Z namespaces/openshift-cluster-csi-drivers/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.119651019Z namespaces/openshift-cluster-csi-drivers/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.119758181Z namespaces/openshift-cluster-csi-drivers/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.119833313Z namespaces/openshift-cluster-csi-drivers/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.120172601Z namespaces/openshift-cluster-csi-drivers/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.120253733Z namespaces/openshift-cluster-csi-drivers/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.120459198Z namespaces/openshift-cluster-csi-drivers/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.120610481Z namespaces/openshift-cluster-csi-drivers/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.120723034Z namespaces/openshift-cluster-csi-drivers/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.120777795Z namespaces/openshift-cluster-csi-drivers/core/serviceaccounts/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.120820686Z namespaces/openshift-cluster-csi-drivers/core/serviceaccounts/aws-ebs-csi-driver-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.120937609Z namespaces/openshift-cluster-csi-drivers/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.12096021Z namespaces/openshift-cluster-csi-drivers/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121053982Z namespaces/openshift-cluster-csi-drivers/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121086282Z namespaces/openshift-cluster-csi-drivers/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121149424Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121195715Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121304888Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121362519Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.1214063Z namespaces/openshift-cluster-csi-drivers/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121445021Z namespaces/openshift-cluster-csi-drivers/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121500412Z namespaces/openshift-cluster-csi-drivers/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121535113Z namespaces/openshift-cluster-csi-drivers/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121600385Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121608605Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121649076Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121799759Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.121977324Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.122112317Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.122335132Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.122579878Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.122779293Z namespaces/openshift-cluster-csi-drivers/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.122791523Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.122836534Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/aws-ebs-csi-driver-node-2t4hs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.122934236Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-driver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.122943386Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-driver/csi-driver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.122952967Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-driver/csi-driver/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123008508Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.12308999Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123247434Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123306935Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-liveness-probe/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123318286Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123328406Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123364916Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123457639Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.1235176Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123557441Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-node-driver-registrar/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123565941Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123570101Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123599202Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123693124Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123768716Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-2t4hs/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123827747Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123858998Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/aws-ebs-csi-driver-node-fvjsj.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123963671Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-driver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123971241Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-driver/csi-driver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.123974931Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-driver/csi-driver/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124020172Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124113634Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124200486Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124244437Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-liveness-probe/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124267128Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124277278Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124306469Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124397401Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124462753Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124512744Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-node-driver-registrar/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124521814Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124528984Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124566155Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124663067Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124730219Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-fvjsj/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.12477793Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124820841Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/aws-ebs-csi-driver-node-gxrpg.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124904913Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-driver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124940924Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-driver/csi-driver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124949804Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-driver/csi-driver/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.124979475Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125076247Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125168139Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125216411Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-liveness-probe/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125234671Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125239701Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125276812Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125370454Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125437786Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125493877Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-node-driver-registrar/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125504377Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125512578Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125541858Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.12563983Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125708272Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-gxrpg/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125748363Z namespaces/openshift-cluster-csi-drivers/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125784824Z namespaces/openshift-cluster-csi-drivers/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125857286Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125869326Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125890196Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/rolebindings/aws-ebs-csi-driver-operator-rolebinding.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125954798Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.125985379Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/roles/aws-ebs-csi-driver-operator-role.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.12605537Z namespaces/openshift-cluster-csi-drivers/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126102352Z namespaces/openshift-cluster-csi-drivers/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126182773Z namespaces/openshift-cluster-machine-approver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126195294Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126199024Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126247335Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/leases/cluster-machine-approver-leader.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126302426Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126311496Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126354327Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/servicemonitors/cluster-machine-approver.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126415239Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126425119Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.12647698Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126632414Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126812398Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.126947651Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.127197067Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.127436573Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.127655368Z namespaces/openshift-cluster-node-tuning-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.127687249Z namespaces/openshift-cluster-node-tuning-operator/openshift-cluster-node-tuning-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.127760191Z namespaces/openshift-cluster-node-tuning-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.127790461Z namespaces/openshift-cluster-node-tuning-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.127861593Z namespaces/openshift-cluster-node-tuning-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.127893064Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128028037Z namespaces/openshift-cluster-node-tuning-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128096979Z namespaces/openshift-cluster-node-tuning-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128208371Z namespaces/openshift-cluster-node-tuning-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128248952Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128289813Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets/tuned.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128369905Z namespaces/openshift-cluster-node-tuning-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128408296Z namespaces/openshift-cluster-node-tuning-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128473088Z namespaces/openshift-cluster-node-tuning-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128513658Z namespaces/openshift-cluster-node-tuning-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128599331Z namespaces/openshift-cluster-node-tuning-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128650912Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128698813Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128775365Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128835756Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128843376Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128879637Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/leases/node-tuning-operator-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.128971999Z namespaces/openshift-cluster-node-tuning-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.12900288Z namespaces/openshift-cluster-node-tuning-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.129912672Z namespaces/openshift-cluster-node-tuning-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.130048975Z namespaces/openshift-cluster-node-tuning-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.130226969Z namespaces/openshift-cluster-node-tuning-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.130302161Z namespaces/openshift-cluster-node-tuning-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.130498085Z namespaces/openshift-cluster-node-tuning-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.130622488Z namespaces/openshift-cluster-node-tuning-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.130834613Z namespaces/openshift-cluster-node-tuning-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.130913125Z namespaces/openshift-cluster-node-tuning-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.130952316Z namespaces/openshift-cluster-node-tuning-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131027398Z namespaces/openshift-cluster-node-tuning-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131063199Z namespaces/openshift-cluster-node-tuning-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131143591Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131256123Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131326355Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131403537Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131470939Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.13153204Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131590071Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131633002Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/prometheusrules/node-tuning-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131690354Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131735455Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors/node-tuning-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131785576Z namespaces/openshift-cluster-node-tuning-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131819487Z namespaces/openshift-cluster-node-tuning-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131869578Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.131879178Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.13193852Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.132151115Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.13236127Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.132503823Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.132707348Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.132983284Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133279882Z namespaces/openshift-cluster-node-tuning-operator/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133290382Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-97kf5/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133336643Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-97kf5/tuned-97kf5.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133438395Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-97kf5/tuned/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133446265Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-97kf5/tuned/tuned/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133451986Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-97kf5/tuned/tuned/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133520087Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-97kf5/tuned/tuned/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.1336295Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-97kf5/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133691111Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-97kf5/tuned/tuned/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133740532Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bl44b/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133776373Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bl44b/tuned-bl44b.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133866515Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bl44b/tuned/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133877876Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bl44b/tuned/tuned/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133907656Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bl44b/tuned/tuned/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.133958487Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bl44b/tuned/tuned/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.13407625Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bl44b/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134170412Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bl44b/tuned/tuned/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134213393Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-n7lt2/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134258505Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-n7lt2/tuned-n7lt2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134346847Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-n7lt2/tuned/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134360807Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-n7lt2/tuned/tuned/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134374787Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-n7lt2/tuned/tuned/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134420718Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-n7lt2/tuned/tuned/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134535211Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-n7lt2/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134604953Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-n7lt2/tuned/tuned/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134741146Z namespaces/openshift-cluster-node-tuning-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134759106Z namespaces/openshift-cluster-node-tuning-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134832748Z namespaces/openshift-cluster-node-tuning-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134861819Z namespaces/openshift-cluster-node-tuning-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134909Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.13491709Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.134951031Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-128-142.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135046653Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-131-162.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135149876Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-139-15.ec2.internal.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135211227Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/tuneds/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135253208Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/tuneds/default.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.13532874Z namespaces/openshift-cluster-samples-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135380831Z namespaces/openshift-cluster-samples-operator/openshift-cluster-samples-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135439533Z namespaces/openshift-cluster-samples-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135484604Z namespaces/openshift-cluster-samples-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135542515Z namespaces/openshift-cluster-samples-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135591326Z namespaces/openshift-cluster-samples-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135682478Z namespaces/openshift-cluster-samples-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135803581Z namespaces/openshift-cluster-samples-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135908444Z namespaces/openshift-cluster-samples-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135958935Z namespaces/openshift-cluster-samples-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.135992136Z namespaces/openshift-cluster-samples-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136036007Z namespaces/openshift-cluster-samples-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136080068Z namespaces/openshift-cluster-samples-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.13618845Z namespaces/openshift-cluster-samples-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136245962Z namespaces/openshift-cluster-samples-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136284043Z namespaces/openshift-cluster-samples-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136363215Z namespaces/openshift-cluster-samples-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136451437Z namespaces/openshift-cluster-samples-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136482517Z namespaces/openshift-cluster-samples-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136665932Z namespaces/openshift-cluster-samples-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136751754Z namespaces/openshift-cluster-samples-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136889177Z namespaces/openshift-cluster-samples-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.136985389Z namespaces/openshift-cluster-samples-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.137108672Z namespaces/openshift-cluster-samples-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.137256345Z namespaces/openshift-cluster-samples-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.137464081Z namespaces/openshift-cluster-samples-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.137524122Z namespaces/openshift-cluster-samples-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.137578473Z namespaces/openshift-cluster-samples-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.137634734Z namespaces/openshift-cluster-samples-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.137682066Z namespaces/openshift-cluster-samples-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.137743117Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.137792018Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.13786944Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.137952162Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138017163Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138065555Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138117066Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138181887Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/prometheusrules/samples-operator-alerts.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138259529Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.13830237Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors/cluster-samples-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138358242Z namespaces/openshift-cluster-samples-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138395872Z namespaces/openshift-cluster-samples-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138443124Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138464004Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138512585Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138668899Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138842043Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.138986797Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.139272383Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.139521529Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.139719894Z namespaces/openshift-cluster-samples-operator/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.139730434Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.139764265Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator-6dc5bdb6b4-k7q4x.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.139842167Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator-watch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.139852067Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator-watch/cluster-samples-operator-watch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.139859527Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.139880728Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.13997287Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140052702Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140078223Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140086513Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator/cluster-samples-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140094013Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator/cluster-samples-operator/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140149034Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator/cluster-samples-operator/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140604195Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator/cluster-samples-operator/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140674266Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-6dc5bdb6b4-k7q4x/cluster-samples-operator/cluster-samples-operator/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140700197Z namespaces/openshift-cluster-samples-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140753019Z namespaces/openshift-cluster-samples-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14080978Z namespaces/openshift-cluster-samples-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14084109Z namespaces/openshift-cluster-samples-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140944923Z namespaces/openshift-cluster-storage-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.140981584Z namespaces/openshift-cluster-storage-operator/openshift-cluster-storage-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141055576Z namespaces/openshift-cluster-storage-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141087546Z namespaces/openshift-cluster-storage-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141176209Z namespaces/openshift-cluster-storage-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14122761Z namespaces/openshift-cluster-storage-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141324252Z namespaces/openshift-cluster-storage-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141417084Z namespaces/openshift-cluster-storage-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141508606Z namespaces/openshift-cluster-storage-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141565388Z namespaces/openshift-cluster-storage-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141614789Z namespaces/openshift-cluster-storage-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14167208Z namespaces/openshift-cluster-storage-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141720501Z namespaces/openshift-cluster-storage-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141798403Z namespaces/openshift-cluster-storage-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141843364Z namespaces/openshift-cluster-storage-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141877775Z namespaces/openshift-cluster-storage-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.141972087Z namespaces/openshift-cluster-storage-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.142015468Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.142030099Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14207597Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/data-source-validator-leader.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.142192253Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/snapshot-controller-leader.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.142254754Z namespaces/openshift-cluster-storage-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.142300975Z namespaces/openshift-cluster-storage-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.142408268Z namespaces/openshift-cluster-storage-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14248937Z namespaces/openshift-cluster-storage-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.142808767Z namespaces/openshift-cluster-storage-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.142886579Z namespaces/openshift-cluster-storage-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.142998062Z namespaces/openshift-cluster-storage-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143147305Z namespaces/openshift-cluster-storage-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143377641Z namespaces/openshift-cluster-storage-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143453932Z namespaces/openshift-cluster-storage-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143530644Z namespaces/openshift-cluster-storage-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143595986Z namespaces/openshift-cluster-storage-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143637357Z namespaces/openshift-cluster-storage-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143692178Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143742769Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143821051Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143901563Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.143966135Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144013326Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144065917Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144109448Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/prometheusrules/prometheus.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144222051Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144253002Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors/cluster-storage-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144325513Z namespaces/openshift-cluster-storage-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144363864Z namespaces/openshift-cluster-storage-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144442496Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144452186Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144509837Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144662111Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.144859676Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.145008419Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.145231825Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.1454692Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.145666215Z namespaces/openshift-cluster-storage-operator/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.145677425Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-7c6cbb6c87-fvtdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.145716786Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-7c6cbb6c87-fvtdb/volume-data-source-validator-7c6cbb6c87-fvtdb.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.145815498Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-7c6cbb6c87-fvtdb/volume-data-source-validator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.145827319Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-7c6cbb6c87-fvtdb/volume-data-source-validator/volume-data-source-validator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.145834349Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-7c6cbb6c87-fvtdb/volume-data-source-validator/volume-data-source-validator/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14587626Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-7c6cbb6c87-fvtdb/volume-data-source-validator/volume-data-source-validator/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.145976932Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-7c6cbb6c87-fvtdb/volume-data-source-validator/volume-data-source-validator/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146044104Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-7c6cbb6c87-fvtdb/volume-data-source-validator/volume-data-source-validator/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146101435Z namespaces/openshift-cluster-storage-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146159656Z namespaces/openshift-cluster-storage-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146213758Z namespaces/openshift-cluster-storage-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146248469Z namespaces/openshift-cluster-storage-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146494044Z namespaces/openshift-cluster-version/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146553226Z namespaces/openshift-cluster-version/openshift-cluster-version.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146614497Z namespaces/openshift-cluster-version/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146656568Z namespaces/openshift-cluster-version/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14671485Z namespaces/openshift-cluster-version/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14674793Z namespaces/openshift-cluster-version/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146839112Z namespaces/openshift-cluster-version/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146912484Z namespaces/openshift-cluster-version/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.146994286Z namespaces/openshift-cluster-version/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147055178Z namespaces/openshift-cluster-version/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147098829Z namespaces/openshift-cluster-version/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147237232Z namespaces/openshift-cluster-version/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147291513Z namespaces/openshift-cluster-version/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147369255Z namespaces/openshift-cluster-version/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147427146Z namespaces/openshift-cluster-version/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147478608Z namespaces/openshift-cluster-version/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14755976Z namespaces/openshift-cluster-version/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147619261Z namespaces/openshift-cluster-version/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147626951Z namespaces/openshift-cluster-version/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147671682Z namespaces/openshift-cluster-version/coordination.k8s.io/leases/version.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147721324Z namespaces/openshift-cluster-version/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147763054Z namespaces/openshift-cluster-version/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147862107Z namespaces/openshift-cluster-version/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.147940709Z namespaces/openshift-cluster-version/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148038971Z namespaces/openshift-cluster-version/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148116513Z namespaces/openshift-cluster-version/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148232116Z namespaces/openshift-cluster-version/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148352108Z namespaces/openshift-cluster-version/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148524092Z namespaces/openshift-cluster-version/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148592014Z namespaces/openshift-cluster-version/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148637505Z namespaces/openshift-cluster-version/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148698867Z namespaces/openshift-cluster-version/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148737337Z namespaces/openshift-cluster-version/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148807309Z namespaces/openshift-cluster-version/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.14884353Z namespaces/openshift-cluster-version/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.148946022Z namespaces/openshift-cluster-version/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.149038415Z namespaces/openshift-cluster-version/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.149092606Z namespaces/openshift-cluster-version/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.149156378Z namespaces/openshift-cluster-version/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.149218369Z namespaces/openshift-cluster-version/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.1492708Z namespaces/openshift-cluster-version/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.149330882Z namespaces/openshift-cluster-version/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.149342742Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.149390003Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.149536596Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.149715311Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.149852114Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.150051289Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.150320235Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.15053219Z namespaces/openshift-cluster-version/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.150577061Z namespaces/openshift-cluster-version/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.150637783Z namespaces/openshift-cluster-version/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.150660153Z namespaces/openshift-cluster-version/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.150772266Z namespaces/openshift-config-managed/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.150815637Z namespaces/openshift-config-managed/openshift-config-managed.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.150873598Z namespaces/openshift-config-managed/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.150913509Z namespaces/openshift-config-managed/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.150978851Z namespaces/openshift-config-managed/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151021162Z namespaces/openshift-config-managed/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151105024Z namespaces/openshift-config-managed/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151216266Z namespaces/openshift-config-managed/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151289218Z namespaces/openshift-config-managed/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151341689Z namespaces/openshift-config-managed/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.15138735Z namespaces/openshift-config-managed/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151447062Z namespaces/openshift-config-managed/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151495333Z namespaces/openshift-config-managed/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151573315Z namespaces/openshift-config-managed/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151618116Z namespaces/openshift-config-managed/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151661897Z namespaces/openshift-config-managed/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151741319Z namespaces/openshift-config-managed/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151820701Z namespaces/openshift-config-managed/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.151866792Z namespaces/openshift-config-managed/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.154979405Z namespaces/openshift-config-managed/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155050337Z namespaces/openshift-config-managed/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155260402Z namespaces/openshift-config-managed/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155329334Z namespaces/openshift-config-managed/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155410756Z namespaces/openshift-config-managed/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155563979Z namespaces/openshift-config-managed/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155670882Z namespaces/openshift-config-managed/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155726883Z namespaces/openshift-config-managed/core/configmaps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155760934Z namespaces/openshift-config-managed/core/configmaps/console-public.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155860856Z namespaces/openshift-config-managed/core/configmaps/openshift-network-features.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155901607Z namespaces/openshift-config-managed/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155939228Z namespaces/openshift-config-managed/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.155982029Z namespaces/openshift-config-managed/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156034861Z namespaces/openshift-config-managed/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156096012Z namespaces/openshift-config-managed/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156157723Z namespaces/openshift-config-managed/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156253015Z namespaces/openshift-config-managed/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156336088Z namespaces/openshift-config-managed/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156391829Z namespaces/openshift-config-managed/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.15642798Z namespaces/openshift-config-managed/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156484511Z namespaces/openshift-config-managed/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156533362Z namespaces/openshift-config-managed/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156573633Z namespaces/openshift-config-managed/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156589903Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156645675Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.156820439Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.157046594Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.157223598Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.157423543Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.15769076Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.157944596Z namespaces/openshift-config-managed/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.157983187Z namespaces/openshift-config-managed/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158047808Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158057008Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158078239Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/rolebindings/openshift-network-public-role-binding.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158163071Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158216852Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/roles/openshift-network-public-role.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158280734Z namespaces/openshift-config-managed/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158297024Z namespaces/openshift-config-managed/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158354555Z namespaces/openshift-config-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158363056Z namespaces/openshift-config-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158371966Z namespaces/openshift-config-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158421667Z namespaces/openshift-config-operator/monitoring.coreos.com/servicemonitors/config-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158513949Z namespaces/openshift-config-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.15853303Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158584581Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158783415Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.158951179Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.159106293Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.159330348Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.159569624Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.15980677Z namespaces/openshift-config/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.159853521Z namespaces/openshift-config/openshift-config.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.159995234Z namespaces/openshift-config/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160035405Z namespaces/openshift-config/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160100307Z namespaces/openshift-config/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160163098Z namespaces/openshift-config/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160254381Z namespaces/openshift-config/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160347513Z namespaces/openshift-config/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160421624Z namespaces/openshift-config/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160482026Z namespaces/openshift-config/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160520767Z namespaces/openshift-config/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160552728Z namespaces/openshift-config/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160613719Z namespaces/openshift-config/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160696331Z namespaces/openshift-config/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160721052Z namespaces/openshift-config/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160772023Z namespaces/openshift-config/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160854685Z namespaces/openshift-config/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160932546Z namespaces/openshift-config/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.160972918Z namespaces/openshift-config/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.161104901Z namespaces/openshift-config/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.161210743Z namespaces/openshift-config/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.161298575Z namespaces/openshift-config/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.161371997Z namespaces/openshift-config/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.161464029Z namespaces/openshift-config/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.161604782Z namespaces/openshift-config/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.161739006Z namespaces/openshift-config/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.161800067Z namespaces/openshift-config/core/secrets/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.161839618Z namespaces/openshift-config/core/secrets/pull-secret.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.1619031Z namespaces/openshift-config/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.161941981Z namespaces/openshift-config/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162011552Z namespaces/openshift-config/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162046303Z namespaces/openshift-config/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162100024Z namespaces/openshift-config/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162156046Z namespaces/openshift-config/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162246468Z namespaces/openshift-config/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.16232555Z namespaces/openshift-config/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162370711Z namespaces/openshift-config/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162404901Z namespaces/openshift-config/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162457463Z namespaces/openshift-config/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162493554Z namespaces/openshift-config/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162535275Z namespaces/openshift-config/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162541775Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162580205Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162736279Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.162918284Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.163050817Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.163332783Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.163582029Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.163778574Z namespaces/openshift-config/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.163809305Z namespaces/openshift-config/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.163866906Z namespaces/openshift-config/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.163900817Z namespaces/openshift-config/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.163991779Z namespaces/openshift-console-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.16403073Z namespaces/openshift-console-operator/openshift-console-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.164233895Z namespaces/openshift-console-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.164262885Z namespaces/openshift-console-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.164345527Z namespaces/openshift-console-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.164385708Z namespaces/openshift-console-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.16447059Z namespaces/openshift-console-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.164594043Z namespaces/openshift-console-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.164695646Z namespaces/openshift-console-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.164724187Z namespaces/openshift-console-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.164777138Z namespaces/openshift-console-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.164839719Z namespaces/openshift-console-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.16488193Z namespaces/openshift-console-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.164978313Z namespaces/openshift-console-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.165045094Z namespaces/openshift-console-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.165066255Z namespaces/openshift-console-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.165189698Z namespaces/openshift-console-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.165230788Z namespaces/openshift-console-operator/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.165237719Z namespaces/openshift-console-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.165270489Z namespaces/openshift-console-operator/coordination.k8s.io/leases/console-operator-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.165352951Z namespaces/openshift-console-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.165393802Z namespaces/openshift-console-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.165981226Z namespaces/openshift-console-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.166069378Z namespaces/openshift-console-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.166342295Z namespaces/openshift-console-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.166418356Z namespaces/openshift-console-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.16654615Z namespaces/openshift-console-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.166669202Z namespaces/openshift-console-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.166849167Z namespaces/openshift-console-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.166906968Z namespaces/openshift-console-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.166940859Z namespaces/openshift-console-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.16699979Z namespaces/openshift-console-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167042361Z namespaces/openshift-console-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167088472Z namespaces/openshift-console-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167147314Z namespaces/openshift-console-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167238606Z namespaces/openshift-console-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167326668Z namespaces/openshift-console-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167383299Z namespaces/openshift-console-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.16742705Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167480092Z namespaces/openshift-console-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167509862Z namespaces/openshift-console-operator/monitoring.coreos.com/prometheusrules/cluster-monitoring-prometheus-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167577554Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167614705Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors/console-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167664436Z namespaces/openshift-console-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167698777Z namespaces/openshift-console-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167744458Z namespaces/openshift-console-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167777839Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.16782604Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.167983704Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.168187788Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.168325742Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.168523286Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.168760332Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.168990647Z namespaces/openshift-console-operator/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.169001698Z namespaces/openshift-console-operator/pods/console-operator-9d4b6777b-c7lrn/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.169035038Z namespaces/openshift-console-operator/pods/console-operator-9d4b6777b-c7lrn/console-operator-9d4b6777b-c7lrn.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.169225813Z namespaces/openshift-console-operator/pods/console-operator-9d4b6777b-c7lrn/console-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.169242363Z namespaces/openshift-console-operator/pods/console-operator-9d4b6777b-c7lrn/console-operator/console-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.169247303Z namespaces/openshift-console-operator/pods/console-operator-9d4b6777b-c7lrn/console-operator/console-operator/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.169285014Z namespaces/openshift-console-operator/pods/console-operator-9d4b6777b-c7lrn/console-operator/console-operator/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.169767806Z namespaces/openshift-console-operator/pods/console-operator-9d4b6777b-c7lrn/console-operator/console-operator/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.169819467Z namespaces/openshift-console-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.169839397Z namespaces/openshift-console-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.169915469Z namespaces/openshift-console-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.16993754Z namespaces/openshift-console-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.170000011Z namespaces/openshift-console-user-settings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.170013482Z namespaces/openshift-console-user-settings/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.170023192Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.170065473Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.170246607Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.170421241Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.170558634Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.170757219Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.170991355Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.171260711Z namespaces/openshift-console/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.171309972Z namespaces/openshift-console/openshift-console.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.171499617Z namespaces/openshift-console/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.171521997Z namespaces/openshift-console/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.171584639Z namespaces/openshift-console/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.17161197Z namespaces/openshift-console/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.171706902Z namespaces/openshift-console/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.171877616Z namespaces/openshift-console/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.172169563Z namespaces/openshift-console/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.172237324Z namespaces/openshift-console/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.172263935Z namespaces/openshift-console/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.172305896Z namespaces/openshift-console/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.172324696Z namespaces/openshift-console/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.172425379Z namespaces/openshift-console/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.17246383Z namespaces/openshift-console/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.172501191Z namespaces/openshift-console/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.172580662Z namespaces/openshift-console/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.172644394Z namespaces/openshift-console/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.172689305Z namespaces/openshift-console/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.17333321Z namespaces/openshift-console/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.173411812Z namespaces/openshift-console/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.173695629Z namespaces/openshift-console/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.173776151Z namespaces/openshift-console/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.173988966Z namespaces/openshift-console/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174111439Z namespaces/openshift-console/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174315264Z namespaces/openshift-console/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174371035Z namespaces/openshift-console/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174417456Z namespaces/openshift-console/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174467037Z namespaces/openshift-console/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174504248Z namespaces/openshift-console/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.17456658Z namespaces/openshift-console/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174645061Z namespaces/openshift-console/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174730103Z namespaces/openshift-console/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174816986Z namespaces/openshift-console/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174870887Z namespaces/openshift-console/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174915348Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.174979039Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.17502834Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors/console.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.175075372Z namespaces/openshift-console/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.175138333Z namespaces/openshift-console/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.175184934Z namespaces/openshift-console/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.175196324Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.175248125Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.175403839Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.175578464Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.175725867Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.175915341Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176165077Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176373192Z namespaces/openshift-console/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176381312Z namespaces/openshift-console/pods/console-8c564f69f-4hd9f/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176412033Z namespaces/openshift-console/pods/console-8c564f69f-4hd9f/console-8c564f69f-4hd9f.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176523676Z namespaces/openshift-console/pods/console-8c564f69f-4hd9f/console/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176531706Z namespaces/openshift-console/pods/console-8c564f69f-4hd9f/console/console/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176537516Z namespaces/openshift-console/pods/console-8c564f69f-4hd9f/console/console/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176578447Z namespaces/openshift-console/pods/console-8c564f69f-4hd9f/console/console/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.17668638Z namespaces/openshift-console/pods/console-8c564f69f-4hd9f/console/console/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176754661Z namespaces/openshift-console/pods/console-8c564f69f-4hd9f/console/console/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176806762Z namespaces/openshift-console/pods/downloads-6bcc868b7-mlhtc/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176846634Z namespaces/openshift-console/pods/downloads-6bcc868b7-mlhtc/downloads-6bcc868b7-mlhtc.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176955756Z namespaces/openshift-console/pods/downloads-6bcc868b7-mlhtc/download-server/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176964516Z namespaces/openshift-console/pods/downloads-6bcc868b7-mlhtc/download-server/download-server/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.176972837Z namespaces/openshift-console/pods/downloads-6bcc868b7-mlhtc/download-server/download-server/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177033418Z namespaces/openshift-console/pods/downloads-6bcc868b7-mlhtc/download-server/download-server/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177356226Z namespaces/openshift-console/pods/downloads-6bcc868b7-mlhtc/download-server/download-server/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177418057Z namespaces/openshift-console/pods/downloads-6bcc868b7-mlhtc/download-server/download-server/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177457568Z namespaces/openshift-console/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177504939Z namespaces/openshift-console/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177579231Z namespaces/openshift-console/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177627692Z namespaces/openshift-console/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177685173Z namespaces/openshift-controller-manager-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177737395Z namespaces/openshift-controller-manager-operator/openshift-controller-manager-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177894848Z namespaces/openshift-controller-manager-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.177936499Z namespaces/openshift-controller-manager-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178000121Z namespaces/openshift-controller-manager-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178039552Z namespaces/openshift-controller-manager-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178143864Z namespaces/openshift-controller-manager-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178226886Z namespaces/openshift-controller-manager-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178306038Z namespaces/openshift-controller-manager-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178363039Z namespaces/openshift-controller-manager-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178410991Z namespaces/openshift-controller-manager-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178465402Z namespaces/openshift-controller-manager-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178532403Z namespaces/openshift-controller-manager-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178595905Z namespaces/openshift-controller-manager-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178656726Z namespaces/openshift-controller-manager-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178697747Z namespaces/openshift-controller-manager-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.17879392Z namespaces/openshift-controller-manager-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178876872Z namespaces/openshift-controller-manager-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.178938913Z namespaces/openshift-controller-manager-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.179093387Z namespaces/openshift-controller-manager-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.17923484Z namespaces/openshift-controller-manager-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.179340603Z namespaces/openshift-controller-manager-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.179436385Z namespaces/openshift-controller-manager-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.179532277Z namespaces/openshift-controller-manager-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.179687431Z namespaces/openshift-controller-manager-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.179891226Z namespaces/openshift-controller-manager-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.179958947Z namespaces/openshift-controller-manager-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.179999148Z namespaces/openshift-controller-manager-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.18005864Z namespaces/openshift-controller-manager-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.1800981Z namespaces/openshift-controller-manager-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.180217483Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.180247154Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.180346977Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.180452179Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.18049014Z namespaces/openshift-controller-manager-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.180534851Z namespaces/openshift-controller-manager-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.180584072Z namespaces/openshift-controller-manager-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.180620233Z namespaces/openshift-controller-manager-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.180683245Z namespaces/openshift-controller-manager-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.180699545Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.180743116Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.18090021Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.181091654Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.181267038Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.181454073Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.181690058Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.181882933Z namespaces/openshift-controller-manager-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.181909894Z namespaces/openshift-controller-manager-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.181962235Z namespaces/openshift-controller-manager-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182008636Z namespaces/openshift-controller-manager-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182084308Z namespaces/openshift-controller-manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182149929Z namespaces/openshift-controller-manager/openshift-controller-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182208521Z namespaces/openshift-controller-manager/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182253242Z namespaces/openshift-controller-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182358214Z namespaces/openshift-controller-manager/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182394035Z namespaces/openshift-controller-manager/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182480477Z namespaces/openshift-controller-manager/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182563599Z namespaces/openshift-controller-manager/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182659041Z namespaces/openshift-controller-manager/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182709902Z namespaces/openshift-controller-manager/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182755923Z namespaces/openshift-controller-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182815475Z namespaces/openshift-controller-manager/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182866776Z namespaces/openshift-controller-manager/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.182963739Z namespaces/openshift-controller-manager/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.18303476Z namespaces/openshift-controller-manager/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183087691Z namespaces/openshift-controller-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183208174Z namespaces/openshift-controller-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183253135Z namespaces/openshift-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183270166Z namespaces/openshift-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183312707Z namespaces/openshift-controller-manager/coordination.k8s.io/leases/openshift-master-controllers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183383438Z namespaces/openshift-controller-manager/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183425179Z namespaces/openshift-controller-manager/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183545762Z namespaces/openshift-controller-manager/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183612474Z namespaces/openshift-controller-manager/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183697436Z namespaces/openshift-controller-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183783038Z namespaces/openshift-controller-manager/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.18385864Z namespaces/openshift-controller-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.183983313Z namespaces/openshift-controller-manager/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184226129Z namespaces/openshift-controller-manager/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184361992Z namespaces/openshift-controller-manager/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184392852Z namespaces/openshift-controller-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184469914Z namespaces/openshift-controller-manager/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184504365Z namespaces/openshift-controller-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184567697Z namespaces/openshift-controller-manager/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184600297Z namespaces/openshift-controller-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184687289Z namespaces/openshift-controller-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184759691Z namespaces/openshift-controller-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184814132Z namespaces/openshift-controller-manager/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184862563Z namespaces/openshift-controller-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184926155Z namespaces/openshift-controller-manager/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.184985526Z namespaces/openshift-controller-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.185034317Z namespaces/openshift-controller-manager/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.185042568Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.185092899Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.185267873Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.185447247Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.185589701Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.185792205Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.186115923Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.186361309Z namespaces/openshift-controller-manager/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.18639223Z namespaces/openshift-controller-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.186471531Z namespaces/openshift-controller-manager/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.186520633Z namespaces/openshift-controller-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.186573804Z namespaces/openshift-dns-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.186621405Z namespaces/openshift-dns-operator/openshift-dns-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.186749338Z namespaces/openshift-dns-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.186780139Z namespaces/openshift-dns-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.18685016Z namespaces/openshift-dns-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.186888681Z namespaces/openshift-dns-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.186973503Z namespaces/openshift-dns-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187052415Z namespaces/openshift-dns-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187158958Z namespaces/openshift-dns-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187220919Z namespaces/openshift-dns-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.18726841Z namespaces/openshift-dns-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187321152Z namespaces/openshift-dns-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187366383Z namespaces/openshift-dns-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187450355Z namespaces/openshift-dns-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187511326Z namespaces/openshift-dns-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187552977Z namespaces/openshift-dns-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187642619Z namespaces/openshift-dns-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187699041Z namespaces/openshift-dns-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187748022Z namespaces/openshift-dns-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187878685Z namespaces/openshift-dns-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.187986718Z namespaces/openshift-dns-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.1881011Z namespaces/openshift-dns-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.188227883Z namespaces/openshift-dns-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.188327155Z namespaces/openshift-dns-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.188463669Z namespaces/openshift-dns-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.188677964Z namespaces/openshift-dns-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.188752136Z namespaces/openshift-dns-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.188808467Z namespaces/openshift-dns-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.188890209Z namespaces/openshift-dns-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.18893721Z namespaces/openshift-dns-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189016702Z namespaces/openshift-dns-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189062743Z namespaces/openshift-dns-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189183416Z namespaces/openshift-dns-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189280928Z namespaces/openshift-dns-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.18935782Z namespaces/openshift-dns-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189401561Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189478633Z namespaces/openshift-dns-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189529354Z namespaces/openshift-dns-operator/monitoring.coreos.com/prometheusrules/dns.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189601216Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189646557Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors/dns-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189711388Z namespaces/openshift-dns-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189752179Z namespaces/openshift-dns-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189824451Z namespaces/openshift-dns-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189844091Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.189897893Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.190087407Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.190319433Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.190467286Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.190691191Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.190942768Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.191323586Z namespaces/openshift-dns-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.191389128Z namespaces/openshift-dns-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.19145064Z namespaces/openshift-dns-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.191490941Z namespaces/openshift-dns-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.191570392Z namespaces/openshift-dns/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.191622594Z namespaces/openshift-dns/openshift-dns.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192236338Z namespaces/openshift-dns/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192287999Z namespaces/openshift-dns/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192378111Z namespaces/openshift-dns/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192432223Z namespaces/openshift-dns/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192598567Z namespaces/openshift-dns/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192688009Z namespaces/openshift-dns/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192776741Z namespaces/openshift-dns/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192822162Z namespaces/openshift-dns/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192878273Z namespaces/openshift-dns/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192962115Z namespaces/openshift-dns/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.192995576Z namespaces/openshift-dns/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.193101939Z namespaces/openshift-dns/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.19317495Z namespaces/openshift-dns/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.193223882Z namespaces/openshift-dns/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.193305133Z namespaces/openshift-dns/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.193364535Z namespaces/openshift-dns/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.193407996Z namespaces/openshift-dns/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.193534109Z namespaces/openshift-dns/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.193620471Z namespaces/openshift-dns/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.193839626Z namespaces/openshift-dns/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.193914648Z namespaces/openshift-dns/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.194232875Z namespaces/openshift-dns/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.194353658Z namespaces/openshift-dns/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.194539213Z namespaces/openshift-dns/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.194607254Z namespaces/openshift-dns/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.194644765Z namespaces/openshift-dns/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.194727087Z namespaces/openshift-dns/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.194766918Z namespaces/openshift-dns/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.19483275Z namespaces/openshift-dns/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.19486805Z namespaces/openshift-dns/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.194957332Z namespaces/openshift-dns/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195025324Z namespaces/openshift-dns/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195093166Z namespaces/openshift-dns/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195166317Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195218669Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.19526794Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors/dns-default.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195314571Z namespaces/openshift-dns/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195355292Z namespaces/openshift-dns/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195401613Z namespaces/openshift-dns/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195414133Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195469105Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195631028Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195808713Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.195945406Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.196168971Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.196409417Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.196641483Z namespaces/openshift-dns/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.196649193Z namespaces/openshift-dns/pods/dns-default-cx2qc/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.196688833Z namespaces/openshift-dns/pods/dns-default-cx2qc/dns-default-cx2qc.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.196780776Z namespaces/openshift-dns/pods/dns-default-cx2qc/dns/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.196788586Z namespaces/openshift-dns/pods/dns-default-cx2qc/dns/dns/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.196792266Z namespaces/openshift-dns/pods/dns-default-cx2qc/dns/dns/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.196822097Z namespaces/openshift-dns/pods/dns-default-cx2qc/dns/dns/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.19694335Z namespaces/openshift-dns/pods/dns-default-cx2qc/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197012021Z namespaces/openshift-dns/pods/dns-default-cx2qc/dns/dns/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197046372Z namespaces/openshift-dns/pods/dns-default-cx2qc/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197054702Z namespaces/openshift-dns/pods/dns-default-cx2qc/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197060182Z namespaces/openshift-dns/pods/dns-default-cx2qc/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197088873Z namespaces/openshift-dns/pods/dns-default-cx2qc/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197211116Z namespaces/openshift-dns/pods/dns-default-cx2qc/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197285718Z namespaces/openshift-dns/pods/dns-default-cx2qc/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197326419Z namespaces/openshift-dns/pods/dns-default-w76jt/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197360949Z namespaces/openshift-dns/pods/dns-default-w76jt/dns-default-w76jt.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197451832Z namespaces/openshift-dns/pods/dns-default-w76jt/dns/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197461082Z namespaces/openshift-dns/pods/dns-default-w76jt/dns/dns/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197466962Z namespaces/openshift-dns/pods/dns-default-w76jt/dns/dns/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197505993Z namespaces/openshift-dns/pods/dns-default-w76jt/dns/dns/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197600245Z namespaces/openshift-dns/pods/dns-default-w76jt/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197668597Z namespaces/openshift-dns/pods/dns-default-w76jt/dns/dns/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197717548Z namespaces/openshift-dns/pods/dns-default-w76jt/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197724988Z namespaces/openshift-dns/pods/dns-default-w76jt/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197736649Z namespaces/openshift-dns/pods/dns-default-w76jt/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197772349Z namespaces/openshift-dns/pods/dns-default-w76jt/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197861251Z namespaces/openshift-dns/pods/dns-default-w76jt/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197939613Z namespaces/openshift-dns/pods/dns-default-w76jt/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.197977184Z namespaces/openshift-dns/pods/dns-default-xxlvr/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198011275Z namespaces/openshift-dns/pods/dns-default-xxlvr/dns-default-xxlvr.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198096647Z namespaces/openshift-dns/pods/dns-default-xxlvr/dns/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198104887Z namespaces/openshift-dns/pods/dns-default-xxlvr/dns/dns/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198111127Z namespaces/openshift-dns/pods/dns-default-xxlvr/dns/dns/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198193719Z namespaces/openshift-dns/pods/dns-default-xxlvr/dns/dns/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198287562Z namespaces/openshift-dns/pods/dns-default-xxlvr/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198354753Z namespaces/openshift-dns/pods/dns-default-xxlvr/dns/dns/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198409044Z namespaces/openshift-dns/pods/dns-default-xxlvr/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198417965Z namespaces/openshift-dns/pods/dns-default-xxlvr/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198427725Z namespaces/openshift-dns/pods/dns-default-xxlvr/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198460015Z namespaces/openshift-dns/pods/dns-default-xxlvr/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198556318Z namespaces/openshift-dns/pods/dns-default-xxlvr/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.19863293Z namespaces/openshift-dns/pods/dns-default-xxlvr/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198682401Z namespaces/openshift-dns/pods/node-resolver-5pvk7/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198727492Z namespaces/openshift-dns/pods/node-resolver-5pvk7/node-resolver-5pvk7.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198817914Z namespaces/openshift-dns/pods/node-resolver-5pvk7/dns-node-resolver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198829904Z namespaces/openshift-dns/pods/node-resolver-5pvk7/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198833744Z namespaces/openshift-dns/pods/node-resolver-5pvk7/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198866875Z namespaces/openshift-dns/pods/node-resolver-5pvk7/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.198959107Z namespaces/openshift-dns/pods/node-resolver-5pvk7/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199026599Z namespaces/openshift-dns/pods/node-resolver-5pvk7/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.19907361Z namespaces/openshift-dns/pods/node-resolver-fmb76/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199116101Z namespaces/openshift-dns/pods/node-resolver-fmb76/node-resolver-fmb76.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199223784Z namespaces/openshift-dns/pods/node-resolver-fmb76/dns-node-resolver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199235344Z namespaces/openshift-dns/pods/node-resolver-fmb76/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199244584Z namespaces/openshift-dns/pods/node-resolver-fmb76/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199281875Z namespaces/openshift-dns/pods/node-resolver-fmb76/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199379647Z namespaces/openshift-dns/pods/node-resolver-fmb76/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199444279Z namespaces/openshift-dns/pods/node-resolver-fmb76/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.19948991Z namespaces/openshift-dns/pods/node-resolver-x6t58/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199534361Z namespaces/openshift-dns/pods/node-resolver-x6t58/node-resolver-x6t58.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199607723Z namespaces/openshift-dns/pods/node-resolver-x6t58/dns-node-resolver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199618063Z namespaces/openshift-dns/pods/node-resolver-x6t58/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199621803Z namespaces/openshift-dns/pods/node-resolver-x6t58/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199673544Z namespaces/openshift-dns/pods/node-resolver-x6t58/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199774417Z namespaces/openshift-dns/pods/node-resolver-x6t58/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199837348Z namespaces/openshift-dns/pods/node-resolver-x6t58/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199873199Z namespaces/openshift-dns/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.1999173Z namespaces/openshift-dns/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.199970551Z namespaces/openshift-dns/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200011152Z namespaces/openshift-dns/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200073294Z namespaces/openshift-etcd/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200142955Z namespaces/openshift-etcd/openshift-etcd.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200282529Z namespaces/openshift-etcd/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20032574Z namespaces/openshift-etcd/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200398821Z namespaces/openshift-etcd/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200443982Z namespaces/openshift-etcd/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200527464Z namespaces/openshift-etcd/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200601966Z namespaces/openshift-etcd/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200681998Z namespaces/openshift-etcd/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200729439Z namespaces/openshift-etcd/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20077319Z namespaces/openshift-etcd/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200819691Z namespaces/openshift-etcd/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200859932Z namespaces/openshift-etcd/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.200959795Z namespaces/openshift-etcd/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.201025776Z namespaces/openshift-etcd/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.201068757Z namespaces/openshift-etcd/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20118445Z namespaces/openshift-etcd/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.201257492Z namespaces/openshift-etcd/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.201304823Z namespaces/openshift-etcd/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.201415296Z namespaces/openshift-etcd/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.201492398Z namespaces/openshift-etcd/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.201571409Z namespaces/openshift-etcd/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.201649051Z namespaces/openshift-etcd/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.201738853Z namespaces/openshift-etcd/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.201860756Z namespaces/openshift-etcd/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20201863Z namespaces/openshift-etcd/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202070211Z namespaces/openshift-etcd/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202116552Z namespaces/openshift-etcd/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202192764Z namespaces/openshift-etcd/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202236575Z namespaces/openshift-etcd/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202290226Z namespaces/openshift-etcd/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202329657Z namespaces/openshift-etcd/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202400979Z namespaces/openshift-etcd/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202510362Z namespaces/openshift-etcd/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202535042Z namespaces/openshift-etcd/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202580633Z namespaces/openshift-etcd/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202640435Z namespaces/openshift-etcd/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202684376Z namespaces/openshift-etcd/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202749127Z namespaces/openshift-etcd/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202768938Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202818549Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.202986543Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.203195908Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.203342781Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.203552216Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.203799402Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204010827Z namespaces/openshift-etcd/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204055528Z namespaces/openshift-etcd/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20413678Z namespaces/openshift-etcd/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204187061Z namespaces/openshift-etcd/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204272183Z namespaces/openshift-host-network/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204319994Z namespaces/openshift-host-network/openshift-host-network.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204397936Z namespaces/openshift-host-network/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204426157Z namespaces/openshift-host-network/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204504999Z namespaces/openshift-host-network/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204534479Z namespaces/openshift-host-network/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204629902Z namespaces/openshift-host-network/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204704563Z namespaces/openshift-host-network/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204783285Z namespaces/openshift-host-network/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204839797Z namespaces/openshift-host-network/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.204886828Z namespaces/openshift-host-network/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20496048Z namespaces/openshift-host-network/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205006621Z namespaces/openshift-host-network/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205093523Z namespaces/openshift-host-network/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205168714Z namespaces/openshift-host-network/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205208445Z namespaces/openshift-host-network/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205303218Z namespaces/openshift-host-network/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205376929Z namespaces/openshift-host-network/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20541851Z namespaces/openshift-host-network/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205521833Z namespaces/openshift-host-network/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205602525Z namespaces/openshift-host-network/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205680697Z namespaces/openshift-host-network/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205761088Z namespaces/openshift-host-network/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20584355Z namespaces/openshift-host-network/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.205987844Z namespaces/openshift-host-network/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206090756Z namespaces/openshift-host-network/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206184489Z namespaces/openshift-host-network/core/resourcequotas/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20622657Z namespaces/openshift-host-network/core/resourcequotas/host-network-namespace-quotas.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206299761Z namespaces/openshift-host-network/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206342502Z namespaces/openshift-host-network/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206403044Z namespaces/openshift-host-network/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206447015Z namespaces/openshift-host-network/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206506026Z namespaces/openshift-host-network/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206552607Z namespaces/openshift-host-network/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206633769Z namespaces/openshift-host-network/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206716581Z namespaces/openshift-host-network/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206772703Z namespaces/openshift-host-network/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206822454Z namespaces/openshift-host-network/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206871395Z namespaces/openshift-host-network/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206911006Z namespaces/openshift-host-network/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.206987488Z namespaces/openshift-host-network/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.207002618Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.207049719Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.207232993Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.207415768Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.207557111Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.207757176Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.207988251Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208237887Z namespaces/openshift-host-network/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208272958Z namespaces/openshift-host-network/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208331189Z namespaces/openshift-host-network/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20836422Z namespaces/openshift-host-network/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208465183Z namespaces/openshift-image-registry/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208505653Z namespaces/openshift-image-registry/openshift-image-registry.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208573385Z namespaces/openshift-image-registry/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208603876Z namespaces/openshift-image-registry/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208671717Z namespaces/openshift-image-registry/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208703428Z namespaces/openshift-image-registry/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208818131Z namespaces/openshift-image-registry/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.208965255Z namespaces/openshift-image-registry/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.20918916Z namespaces/openshift-image-registry/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209228671Z namespaces/openshift-image-registry/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209267522Z namespaces/openshift-image-registry/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209324573Z namespaces/openshift-image-registry/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209358794Z namespaces/openshift-image-registry/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209469006Z namespaces/openshift-image-registry/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209531688Z namespaces/openshift-image-registry/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209570669Z namespaces/openshift-image-registry/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209655831Z namespaces/openshift-image-registry/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209706682Z namespaces/openshift-image-registry/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209713172Z namespaces/openshift-image-registry/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209755453Z namespaces/openshift-image-registry/coordination.k8s.io/leases/openshift-master-controllers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209807124Z namespaces/openshift-image-registry/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.209842075Z namespaces/openshift-image-registry/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.210794728Z namespaces/openshift-image-registry/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21089068Z namespaces/openshift-image-registry/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.211195477Z namespaces/openshift-image-registry/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.211281929Z namespaces/openshift-image-registry/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.211491894Z namespaces/openshift-image-registry/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.211623307Z namespaces/openshift-image-registry/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.211821232Z namespaces/openshift-image-registry/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.211881133Z namespaces/openshift-image-registry/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.211936605Z namespaces/openshift-image-registry/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.211979206Z namespaces/openshift-image-registry/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212031657Z namespaces/openshift-image-registry/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212063038Z namespaces/openshift-image-registry/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212098359Z namespaces/openshift-image-registry/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212223252Z namespaces/openshift-image-registry/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212308044Z namespaces/openshift-image-registry/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212378615Z namespaces/openshift-image-registry/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212422517Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212494958Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21257566Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/image-registry-operator-alerts.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212683453Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/image-registry-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212771455Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/imagestreams-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212847047Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212881657Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors/image-registry.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.212967029Z namespaces/openshift-image-registry/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21298712Z namespaces/openshift-image-registry/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.213054771Z namespaces/openshift-image-registry/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.213068702Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.213116613Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.213354799Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.213548643Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.213723417Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.213932782Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.214207599Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.214425994Z namespaces/openshift-image-registry/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.214435344Z namespaces/openshift-image-registry/pods/image-registry-5595d687-bktl2/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.214479845Z namespaces/openshift-image-registry/pods/image-registry-5595d687-bktl2/image-registry-5595d687-bktl2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.214593948Z namespaces/openshift-image-registry/pods/image-registry-5595d687-bktl2/registry/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.214601828Z namespaces/openshift-image-registry/pods/image-registry-5595d687-bktl2/registry/registry/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.214616398Z namespaces/openshift-image-registry/pods/image-registry-5595d687-bktl2/registry/registry/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.214658049Z namespaces/openshift-image-registry/pods/image-registry-5595d687-bktl2/registry/registry/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.215673173Z namespaces/openshift-image-registry/pods/image-registry-5595d687-bktl2/registry/registry/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.215745945Z namespaces/openshift-image-registry/pods/image-registry-5595d687-bktl2/registry/registry/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.215786216Z namespaces/openshift-image-registry/pods/node-ca-2g4mv/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.215826677Z namespaces/openshift-image-registry/pods/node-ca-2g4mv/node-ca-2g4mv.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.215919689Z namespaces/openshift-image-registry/pods/node-ca-2g4mv/node-ca/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.215928619Z namespaces/openshift-image-registry/pods/node-ca-2g4mv/node-ca/node-ca/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21593464Z namespaces/openshift-image-registry/pods/node-ca-2g4mv/node-ca/node-ca/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.215978081Z namespaces/openshift-image-registry/pods/node-ca-2g4mv/node-ca/node-ca/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216112184Z namespaces/openshift-image-registry/pods/node-ca-2g4mv/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216222256Z namespaces/openshift-image-registry/pods/node-ca-2g4mv/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216258527Z namespaces/openshift-image-registry/pods/node-ca-nwpfl/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216303068Z namespaces/openshift-image-registry/pods/node-ca-nwpfl/node-ca-nwpfl.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21637275Z namespaces/openshift-image-registry/pods/node-ca-nwpfl/node-ca/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21637914Z namespaces/openshift-image-registry/pods/node-ca-nwpfl/node-ca/node-ca/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21638276Z namespaces/openshift-image-registry/pods/node-ca-nwpfl/node-ca/node-ca/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216423891Z namespaces/openshift-image-registry/pods/node-ca-nwpfl/node-ca/node-ca/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216580985Z namespaces/openshift-image-registry/pods/node-ca-nwpfl/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216649377Z namespaces/openshift-image-registry/pods/node-ca-nwpfl/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216699598Z namespaces/openshift-image-registry/pods/node-ca-r5z5s/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216746339Z namespaces/openshift-image-registry/pods/node-ca-r5z5s/node-ca-r5z5s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21681457Z namespaces/openshift-image-registry/pods/node-ca-r5z5s/node-ca/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216821261Z namespaces/openshift-image-registry/pods/node-ca-r5z5s/node-ca/node-ca/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216825341Z namespaces/openshift-image-registry/pods/node-ca-r5z5s/node-ca/node-ca/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.216863831Z namespaces/openshift-image-registry/pods/node-ca-r5z5s/node-ca/node-ca/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217008235Z namespaces/openshift-image-registry/pods/node-ca-r5z5s/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217080007Z namespaces/openshift-image-registry/pods/node-ca-r5z5s/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217136408Z namespaces/openshift-image-registry/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21722587Z namespaces/openshift-image-registry/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217286952Z namespaces/openshift-image-registry/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217321632Z namespaces/openshift-image-registry/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217374464Z namespaces/openshift-infra/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217380874Z namespaces/openshift-infra/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217401214Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217449576Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217608679Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217789893Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.217928797Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.218146972Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.218395898Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.218616403Z namespaces/openshift-ingress-canary/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.218651254Z namespaces/openshift-ingress-canary/openshift-ingress-canary.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.218724056Z namespaces/openshift-ingress-canary/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.218764727Z namespaces/openshift-ingress-canary/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.218831838Z namespaces/openshift-ingress-canary/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.218862639Z namespaces/openshift-ingress-canary/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.218975892Z namespaces/openshift-ingress-canary/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219055434Z namespaces/openshift-ingress-canary/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219163896Z namespaces/openshift-ingress-canary/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219213587Z namespaces/openshift-ingress-canary/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219262668Z namespaces/openshift-ingress-canary/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21931116Z namespaces/openshift-ingress-canary/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.21934286Z namespaces/openshift-ingress-canary/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219435152Z namespaces/openshift-ingress-canary/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219467243Z namespaces/openshift-ingress-canary/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219519874Z namespaces/openshift-ingress-canary/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219608967Z namespaces/openshift-ingress-canary/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219671608Z namespaces/openshift-ingress-canary/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219717879Z namespaces/openshift-ingress-canary/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219825482Z namespaces/openshift-ingress-canary/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.219916064Z namespaces/openshift-ingress-canary/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.220064278Z namespaces/openshift-ingress-canary/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22017273Z namespaces/openshift-ingress-canary/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.220348994Z namespaces/openshift-ingress-canary/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.220473197Z namespaces/openshift-ingress-canary/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.220654672Z namespaces/openshift-ingress-canary/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.220785115Z namespaces/openshift-ingress-canary/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.220831326Z namespaces/openshift-ingress-canary/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.220894057Z namespaces/openshift-ingress-canary/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.220929288Z namespaces/openshift-ingress-canary/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22100486Z namespaces/openshift-ingress-canary/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22103347Z namespaces/openshift-ingress-canary/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.221140623Z namespaces/openshift-ingress-canary/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.221231015Z namespaces/openshift-ingress-canary/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.221285906Z namespaces/openshift-ingress-canary/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.221331257Z namespaces/openshift-ingress-canary/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.221381109Z namespaces/openshift-ingress-canary/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22142885Z namespaces/openshift-ingress-canary/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.221472451Z namespaces/openshift-ingress-canary/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.221490311Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.221540382Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.221698846Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.221880421Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.222020054Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.222243139Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.222519266Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.222736721Z namespaces/openshift-ingress-canary/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.222747451Z namespaces/openshift-ingress-canary/pods/ingress-canary-7tfnv/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.222785972Z namespaces/openshift-ingress-canary/pods/ingress-canary-7tfnv/ingress-canary-7tfnv.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223067749Z namespaces/openshift-ingress-canary/pods/ingress-canary-7tfnv/serve-healthcheck-canary/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223077179Z namespaces/openshift-ingress-canary/pods/ingress-canary-7tfnv/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223082009Z namespaces/openshift-ingress-canary/pods/ingress-canary-7tfnv/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22314918Z namespaces/openshift-ingress-canary/pods/ingress-canary-7tfnv/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223248703Z namespaces/openshift-ingress-canary/pods/ingress-canary-7tfnv/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223321905Z namespaces/openshift-ingress-canary/pods/ingress-canary-7tfnv/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223372366Z namespaces/openshift-ingress-canary/pods/ingress-canary-gdwvv/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223421597Z namespaces/openshift-ingress-canary/pods/ingress-canary-gdwvv/ingress-canary-gdwvv.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223492279Z namespaces/openshift-ingress-canary/pods/ingress-canary-gdwvv/serve-healthcheck-canary/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223500209Z namespaces/openshift-ingress-canary/pods/ingress-canary-gdwvv/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223520269Z namespaces/openshift-ingress-canary/pods/ingress-canary-gdwvv/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22354325Z namespaces/openshift-ingress-canary/pods/ingress-canary-gdwvv/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223647352Z namespaces/openshift-ingress-canary/pods/ingress-canary-gdwvv/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223725414Z namespaces/openshift-ingress-canary/pods/ingress-canary-gdwvv/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223775275Z namespaces/openshift-ingress-canary/pods/ingress-canary-pxf27/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223821856Z namespaces/openshift-ingress-canary/pods/ingress-canary-pxf27/ingress-canary-pxf27.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223895878Z namespaces/openshift-ingress-canary/pods/ingress-canary-pxf27/serve-healthcheck-canary/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223902428Z namespaces/openshift-ingress-canary/pods/ingress-canary-pxf27/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.223908238Z namespaces/openshift-ingress-canary/pods/ingress-canary-pxf27/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22395431Z namespaces/openshift-ingress-canary/pods/ingress-canary-pxf27/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224051232Z namespaces/openshift-ingress-canary/pods/ingress-canary-pxf27/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224140314Z namespaces/openshift-ingress-canary/pods/ingress-canary-pxf27/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224223846Z namespaces/openshift-ingress-canary/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224278417Z namespaces/openshift-ingress-canary/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224343689Z namespaces/openshift-ingress-canary/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224373769Z namespaces/openshift-ingress-canary/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224491102Z namespaces/openshift-ingress-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224537493Z namespaces/openshift-ingress-operator/openshift-ingress-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224612395Z namespaces/openshift-ingress-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224655466Z namespaces/openshift-ingress-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224721848Z namespaces/openshift-ingress-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224762869Z namespaces/openshift-ingress-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224850221Z namespaces/openshift-ingress-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.224942003Z namespaces/openshift-ingress-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.225027695Z namespaces/openshift-ingress-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.225065536Z namespaces/openshift-ingress-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.225114177Z namespaces/openshift-ingress-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.225209249Z namespaces/openshift-ingress-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22523112Z namespaces/openshift-ingress-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.225326652Z namespaces/openshift-ingress-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.225386973Z namespaces/openshift-ingress-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.225420754Z namespaces/openshift-ingress-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.225506686Z namespaces/openshift-ingress-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.225578028Z namespaces/openshift-ingress-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.225621209Z namespaces/openshift-ingress-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.226224583Z namespaces/openshift-ingress-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.226303555Z namespaces/openshift-ingress-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.226413258Z namespaces/openshift-ingress-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22649422Z namespaces/openshift-ingress-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.226574361Z namespaces/openshift-ingress-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.226701945Z namespaces/openshift-ingress-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22691531Z namespaces/openshift-ingress-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.226972001Z namespaces/openshift-ingress-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227015912Z namespaces/openshift-ingress-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227067863Z namespaces/openshift-ingress-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227115625Z namespaces/openshift-ingress-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227185396Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227193076Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/dnsrecords/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227247748Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/dnsrecords/default-wildcard.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227319769Z namespaces/openshift-ingress-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.2273599Z namespaces/openshift-ingress-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227453342Z namespaces/openshift-ingress-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227565325Z namespaces/openshift-ingress-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227636377Z namespaces/openshift-ingress-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227681508Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227734239Z namespaces/openshift-ingress-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22777719Z namespaces/openshift-ingress-operator/monitoring.coreos.com/prometheusrules/ingress-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227849432Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227890843Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors/ingress-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.227962554Z namespaces/openshift-ingress-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.228001106Z namespaces/openshift-ingress-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.228060197Z namespaces/openshift-ingress-operator/operator.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.228068417Z namespaces/openshift-ingress-operator/operator.openshift.io/ingresscontrollers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.228096418Z namespaces/openshift-ingress-operator/operator.openshift.io/ingresscontrollers/default.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.228214891Z namespaces/openshift-ingress-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.228232551Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.228284302Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.228446646Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.228655951Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.228814325Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22902931Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.229292736Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.229511461Z namespaces/openshift-ingress-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.229547032Z namespaces/openshift-ingress-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.229616444Z namespaces/openshift-ingress-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.229661335Z namespaces/openshift-ingress-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.229748797Z namespaces/openshift-ingress/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.229802698Z namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.22986974Z namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.229918821Z namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.229984902Z namespaces/openshift-ingress/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.230027923Z namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.230103845Z namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.230369951Z namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.230613957Z namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.230670419Z namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23070789Z namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.230774891Z namespaces/openshift-ingress/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.230817712Z namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.230903934Z namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.230959325Z namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.230997426Z namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.231089139Z namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23115492Z namespaces/openshift-ingress/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.231174111Z namespaces/openshift-ingress/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.231212331Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-ca-openshift-gateway.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.231306364Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-deployment-openshift-gateway.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.231395976Z namespaces/openshift-ingress/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.231441667Z namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.231871897Z namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23197553Z namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.232247486Z namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.232318318Z namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.232564654Z namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.232692797Z namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.232915252Z namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233019094Z namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233051375Z namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233147097Z namespaces/openshift-ingress/gateway.networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233220929Z namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23324077Z namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/openshift-ai-inference.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233352392Z namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233367582Z namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233452904Z namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233471335Z namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233571997Z namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233645449Z namespaces/openshift-ingress/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23368704Z namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233717681Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233781502Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233821303Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors/router-default.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233862464Z namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233897875Z namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233949236Z namespaces/openshift-ingress/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.233971937Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.234018388Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.234210742Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.234395067Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23452883Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.234731505Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.234969551Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23536249Z namespaces/openshift-ingress/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23537438Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.235430091Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/istiod-openshift-gateway-55ff986f96-rgd49.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.235542384Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.235553294Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/discovery/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.235562794Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/discovery/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.235600375Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/discovery/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.237407108Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23747793Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-55ff986f96-rgd49/discovery/discovery/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.237529001Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.237563062Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.237694935Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.237703405Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/istio-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.237707985Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.237752066Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23790848Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/istio-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.237973052Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-7c5447bb76-q9nvg/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238016343Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238073034Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router-default-5fdf56dbd-s82zg.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238198027Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238208707Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/router/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238221388Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/router/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238270979Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/router/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238386972Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/router/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238457593Z namespaces/openshift-ingress/pods/router-default-5fdf56dbd-s82zg/router/router/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238486224Z namespaces/openshift-ingress/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238520665Z namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238582656Z namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238610227Z namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23872216Z namespaces/openshift-insights/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238772461Z namespaces/openshift-insights/openshift-insights.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238834982Z namespaces/openshift-insights/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238876083Z namespaces/openshift-insights/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238947915Z namespaces/openshift-insights/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.238991486Z namespaces/openshift-insights/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239145719Z namespaces/openshift-insights/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239265202Z namespaces/openshift-insights/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239367895Z namespaces/openshift-insights/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239426026Z namespaces/openshift-insights/apps/deployments/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239473837Z namespaces/openshift-insights/apps/deployments/insights-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239550839Z namespaces/openshift-insights/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.23959035Z namespaces/openshift-insights/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239653302Z namespaces/openshift-insights/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239695392Z namespaces/openshift-insights/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239782005Z namespaces/openshift-insights/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239831576Z namespaces/openshift-insights/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239876857Z namespaces/openshift-insights/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.239958789Z namespaces/openshift-insights/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.240038741Z namespaces/openshift-insights/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.240089372Z namespaces/openshift-insights/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.241052565Z namespaces/openshift-insights/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.241197658Z namespaces/openshift-insights/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.241462114Z namespaces/openshift-insights/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.241592468Z namespaces/openshift-insights/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.241912255Z namespaces/openshift-insights/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242002497Z namespaces/openshift-insights/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242181451Z namespaces/openshift-insights/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242264903Z namespaces/openshift-insights/core/configmaps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242314445Z namespaces/openshift-insights/core/configmaps/service-ca-bundle.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242396567Z namespaces/openshift-insights/core/serviceaccounts/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242435948Z namespaces/openshift-insights/core/serviceaccounts/gather.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.24252939Z namespaces/openshift-insights/core/serviceaccounts/operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242577291Z namespaces/openshift-insights/core/services/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242621352Z namespaces/openshift-insights/core/services/metrics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242674873Z namespaces/openshift-insights/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242714354Z namespaces/openshift-insights/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242802686Z namespaces/openshift-insights/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242844337Z namespaces/openshift-insights/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242899478Z namespaces/openshift-insights/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.242938829Z namespaces/openshift-insights/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243028681Z namespaces/openshift-insights/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243109783Z namespaces/openshift-insights/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243177775Z namespaces/openshift-insights/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243220676Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243295038Z namespaces/openshift-insights/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243339379Z namespaces/openshift-insights/monitoring.coreos.com/prometheusrules/insights-prometheus-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.24341558Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243460161Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors/insights-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243530883Z namespaces/openshift-insights/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243573284Z namespaces/openshift-insights/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243627356Z namespaces/openshift-insights/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243636066Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243685967Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.243879211Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.244108057Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.244307512Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.244523057Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.244782983Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.2450641Z namespaces/openshift-insights/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.2450746Z namespaces/openshift-insights/pods/insights-operator-585dfdc468-xfzqb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.245103691Z namespaces/openshift-insights/pods/insights-operator-585dfdc468-xfzqb/insights-operator-585dfdc468-xfzqb.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.245260324Z namespaces/openshift-insights/pods/insights-operator-585dfdc468-xfzqb/insights-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.245274365Z namespaces/openshift-insights/pods/insights-operator-585dfdc468-xfzqb/insights-operator/insights-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.245279745Z namespaces/openshift-insights/pods/insights-operator-585dfdc468-xfzqb/insights-operator/insights-operator/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.245317906Z namespaces/openshift-insights/pods/insights-operator-585dfdc468-xfzqb/insights-operator/insights-operator/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.245693605Z namespaces/openshift-insights/pods/insights-operator-585dfdc468-xfzqb/insights-operator/insights-operator/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246008672Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246044463Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/insights-runtime-extractor-2mqlt.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246155945Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246177716Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/exporter/exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246188136Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/exporter/exporter/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246244448Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/exporter/exporter/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.24633408Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246407142Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/exporter/exporter/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246439062Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/extractor/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246446652Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/extractor/extractor/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246500094Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/extractor/extractor/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246540965Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/extractor/extractor/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246648877Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246720449Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/extractor/extractor/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.24676371Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.24678287Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.24678814Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246809631Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246900383Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.246977505Z namespaces/openshift-insights/pods/insights-runtime-extractor-2mqlt/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247071447Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247145799Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/insights-runtime-extractor-5dm7q.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247242341Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247254442Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/exporter/exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247259152Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/exporter/exporter/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247286232Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/exporter/exporter/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247377105Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247449556Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/exporter/exporter/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247488457Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/extractor/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247499207Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/extractor/extractor/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247504168Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/extractor/extractor/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247552049Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/extractor/extractor/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247655731Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247728353Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/extractor/extractor/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247767404Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247776054Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247788674Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247825005Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247916807Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.247989969Z namespaces/openshift-insights/pods/insights-runtime-extractor-5dm7q/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.24804642Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248095002Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/insights-runtime-extractor-qbgk9.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248268595Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248293666Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/exporter/exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.24846725Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/exporter/exporter/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248510051Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/exporter/exporter/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248606003Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248676895Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/exporter/exporter/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248714036Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/extractor/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248720586Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/extractor/extractor/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248724366Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/extractor/extractor/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248766587Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/extractor/extractor/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.24886743Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248937082Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/extractor/extractor/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.248990763Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249000063Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249003823Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249048274Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249162007Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249232738Z namespaces/openshift-insights/pods/insights-runtime-extractor-qbgk9/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249274919Z namespaces/openshift-insights/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249318931Z namespaces/openshift-insights/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249377872Z namespaces/openshift-insights/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249426973Z namespaces/openshift-insights/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249471684Z namespaces/openshift-keda/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249479474Z namespaces/openshift-keda/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249486095Z namespaces/openshift-keda/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249536656Z namespaces/openshift-keda/coordination.k8s.io/leases/olm-operator.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249638478Z namespaces/openshift-keda/coordination.k8s.io/leases/operator.keda.sh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.2496962Z namespaces/openshift-keda/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.24970408Z namespaces/openshift-keda/monitoring.coreos.com/podmonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249746531Z namespaces/openshift-keda/monitoring.coreos.com/podmonitors/keda-olm-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249823152Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249869704Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-admission-webhooks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.249963366Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-metrics-apiserver.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.250045748Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.25014134Z namespaces/openshift-keda/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.2501631Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.250197721Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.250357315Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.25054874Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/custom-metrics-autoscaler.v2.18.1-2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.250738124Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.250934689Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.251077572Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.251313448Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.251510732Z namespaces/openshift-keda/operators.coreos.com/installplans/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.251558813Z namespaces/openshift-keda/operators.coreos.com/installplans/install-sjhxl.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.251654376Z namespaces/openshift-keda/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.251674426Z namespaces/openshift-keda/operators.coreos.com/operatorconditions/custom-metrics-autoscaler.v2.18.1-2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.251744488Z namespaces/openshift-keda/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.251793549Z namespaces/openshift-keda/operators.coreos.com/operatorgroups/openshift-keda.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.25184784Z namespaces/openshift-keda/operators.coreos.com/subscriptions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.251890932Z namespaces/openshift-keda/operators.coreos.com/subscriptions/openshift-custom-metrics-autoscaler-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.251987674Z namespaces/openshift-kube-apiserver-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252030485Z namespaces/openshift-kube-apiserver-operator/openshift-kube-apiserver-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252099866Z namespaces/openshift-kube-apiserver-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252158218Z namespaces/openshift-kube-apiserver-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252227869Z namespaces/openshift-kube-apiserver-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252278131Z namespaces/openshift-kube-apiserver-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252366843Z namespaces/openshift-kube-apiserver-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252446674Z namespaces/openshift-kube-apiserver-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252525627Z namespaces/openshift-kube-apiserver-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252576998Z namespaces/openshift-kube-apiserver-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252620149Z namespaces/openshift-kube-apiserver-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.25267219Z namespaces/openshift-kube-apiserver-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252718721Z namespaces/openshift-kube-apiserver-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252803593Z namespaces/openshift-kube-apiserver-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252862334Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.252914436Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.253011138Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.25308566Z namespaces/openshift-kube-apiserver-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.253138371Z namespaces/openshift-kube-apiserver-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.253265614Z namespaces/openshift-kube-apiserver-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.253344796Z namespaces/openshift-kube-apiserver-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.253431468Z namespaces/openshift-kube-apiserver-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.2535223Z namespaces/openshift-kube-apiserver-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.253601762Z namespaces/openshift-kube-apiserver-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.253719845Z namespaces/openshift-kube-apiserver-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.253898689Z namespaces/openshift-kube-apiserver-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.25396483Z namespaces/openshift-kube-apiserver-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254004462Z namespaces/openshift-kube-apiserver-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254064813Z namespaces/openshift-kube-apiserver-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254106624Z namespaces/openshift-kube-apiserver-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254187286Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254236647Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254323199Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254407501Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254468502Z namespaces/openshift-kube-apiserver-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254516654Z namespaces/openshift-kube-apiserver-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254580175Z namespaces/openshift-kube-apiserver-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254624036Z namespaces/openshift-kube-apiserver-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254668557Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254678947Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254742409Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.254893683Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.255073597Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.255237191Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.255443636Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.255682281Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.255898606Z namespaces/openshift-kube-apiserver-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.255942807Z namespaces/openshift-kube-apiserver-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256012519Z namespaces/openshift-kube-apiserver-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.2560578Z namespaces/openshift-kube-apiserver-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256152842Z namespaces/openshift-kube-apiserver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256208534Z namespaces/openshift-kube-apiserver/openshift-kube-apiserver.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256262005Z namespaces/openshift-kube-apiserver/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256316426Z namespaces/openshift-kube-apiserver/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256383148Z namespaces/openshift-kube-apiserver/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256425909Z namespaces/openshift-kube-apiserver/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256513291Z namespaces/openshift-kube-apiserver/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256593503Z namespaces/openshift-kube-apiserver/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256681665Z namespaces/openshift-kube-apiserver/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256737866Z namespaces/openshift-kube-apiserver/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256778687Z namespaces/openshift-kube-apiserver/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256846959Z namespaces/openshift-kube-apiserver/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.2568864Z namespaces/openshift-kube-apiserver/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.256994062Z namespaces/openshift-kube-apiserver/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257043303Z namespaces/openshift-kube-apiserver/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257089044Z namespaces/openshift-kube-apiserver/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257239078Z namespaces/openshift-kube-apiserver/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257376391Z namespaces/openshift-kube-apiserver/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257422763Z namespaces/openshift-kube-apiserver/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257528565Z namespaces/openshift-kube-apiserver/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257610007Z namespaces/openshift-kube-apiserver/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257695579Z namespaces/openshift-kube-apiserver/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257784831Z namespaces/openshift-kube-apiserver/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257864883Z namespaces/openshift-kube-apiserver/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.257985636Z namespaces/openshift-kube-apiserver/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.25816027Z namespaces/openshift-kube-apiserver/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258213421Z namespaces/openshift-kube-apiserver/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258258362Z namespaces/openshift-kube-apiserver/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258308424Z namespaces/openshift-kube-apiserver/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258339324Z namespaces/openshift-kube-apiserver/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258410216Z namespaces/openshift-kube-apiserver/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258448037Z namespaces/openshift-kube-apiserver/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258536589Z namespaces/openshift-kube-apiserver/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258614271Z namespaces/openshift-kube-apiserver/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258671522Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258712813Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258798935Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258854536Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/api-usage.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.258952199Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/podsecurity.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.25900745Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.259057721Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors/openshift-kube-apiserver.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.259116893Z namespaces/openshift-kube-apiserver/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.259169324Z namespaces/openshift-kube-apiserver/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.259247246Z namespaces/openshift-kube-apiserver/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.259255026Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.259294067Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.259456781Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.259640535Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.259780548Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.259979983Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.260239809Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.260447984Z namespaces/openshift-kube-apiserver/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.260471175Z namespaces/openshift-kube-apiserver/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.260552077Z namespaces/openshift-kube-apiserver/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.260596238Z namespaces/openshift-kube-apiserver/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26071565Z namespaces/openshift-kube-controller-manager-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.260759351Z namespaces/openshift-kube-controller-manager-operator/openshift-kube-controller-manager-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.260831133Z namespaces/openshift-kube-controller-manager-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.260867344Z namespaces/openshift-kube-controller-manager-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.260951076Z namespaces/openshift-kube-controller-manager-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.260987767Z namespaces/openshift-kube-controller-manager-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261080919Z namespaces/openshift-kube-controller-manager-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261206502Z namespaces/openshift-kube-controller-manager-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261283124Z namespaces/openshift-kube-controller-manager-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261419977Z namespaces/openshift-kube-controller-manager-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261457258Z namespaces/openshift-kube-controller-manager-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261511289Z namespaces/openshift-kube-controller-manager-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26155252Z namespaces/openshift-kube-controller-manager-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261636792Z namespaces/openshift-kube-controller-manager-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261687944Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261720904Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261810636Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261863798Z namespaces/openshift-kube-controller-manager-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.261896838Z namespaces/openshift-kube-controller-manager-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.262013711Z namespaces/openshift-kube-controller-manager-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.262097853Z namespaces/openshift-kube-controller-manager-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.262261167Z namespaces/openshift-kube-controller-manager-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.262346539Z namespaces/openshift-kube-controller-manager-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.262423701Z namespaces/openshift-kube-controller-manager-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.262559014Z namespaces/openshift-kube-controller-manager-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.262753049Z namespaces/openshift-kube-controller-manager-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26281064Z namespaces/openshift-kube-controller-manager-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.262846421Z namespaces/openshift-kube-controller-manager-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.262907062Z namespaces/openshift-kube-controller-manager-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.262944303Z namespaces/openshift-kube-controller-manager-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263006395Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263048896Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263163538Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26324019Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263300832Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263327062Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263390984Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263436945Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors/kube-controller-manager-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263498116Z namespaces/openshift-kube-controller-manager-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263537287Z namespaces/openshift-kube-controller-manager-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263591609Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263600379Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26364662Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263803034Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.263981928Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.264148642Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.264361917Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.264604433Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.264804447Z namespaces/openshift-kube-controller-manager-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.264833698Z namespaces/openshift-kube-controller-manager-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.264898229Z namespaces/openshift-kube-controller-manager-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26493897Z namespaces/openshift-kube-controller-manager-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265018292Z namespaces/openshift-kube-controller-manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265058613Z namespaces/openshift-kube-controller-manager/openshift-kube-controller-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265136505Z namespaces/openshift-kube-controller-manager/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265171666Z namespaces/openshift-kube-controller-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265237737Z namespaces/openshift-kube-controller-manager/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265285749Z namespaces/openshift-kube-controller-manager/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265365851Z namespaces/openshift-kube-controller-manager/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265447173Z namespaces/openshift-kube-controller-manager/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265528915Z namespaces/openshift-kube-controller-manager/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265579916Z namespaces/openshift-kube-controller-manager/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265623547Z namespaces/openshift-kube-controller-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265679648Z namespaces/openshift-kube-controller-manager/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265730959Z namespaces/openshift-kube-controller-manager/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265808331Z namespaces/openshift-kube-controller-manager/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265854762Z namespaces/openshift-kube-controller-manager/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265895733Z namespaces/openshift-kube-controller-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.265981735Z namespaces/openshift-kube-controller-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.266033247Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.266047687Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.266090428Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/leases/cluster-policy-controller-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26619348Z namespaces/openshift-kube-controller-manager/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.266232371Z namespaces/openshift-kube-controller-manager/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.266337174Z namespaces/openshift-kube-controller-manager/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.266416696Z namespaces/openshift-kube-controller-manager/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.266709612Z namespaces/openshift-kube-controller-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.266800105Z namespaces/openshift-kube-controller-manager/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.266883007Z namespaces/openshift-kube-controller-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26700345Z namespaces/openshift-kube-controller-manager/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267179143Z namespaces/openshift-kube-controller-manager/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267245345Z namespaces/openshift-kube-controller-manager/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267284496Z namespaces/openshift-kube-controller-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267341918Z namespaces/openshift-kube-controller-manager/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267397869Z namespaces/openshift-kube-controller-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26745916Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267501441Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267587803Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267669295Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267727327Z namespaces/openshift-kube-controller-manager/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267766408Z namespaces/openshift-kube-controller-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267821899Z namespaces/openshift-kube-controller-manager/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26786841Z namespaces/openshift-kube-controller-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267925701Z namespaces/openshift-kube-controller-manager/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267941362Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.267988253Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.268155897Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.268357011Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.268497915Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.268692839Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.268938015Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26914102Z namespaces/openshift-kube-controller-manager/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269194791Z namespaces/openshift-kube-controller-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269241943Z namespaces/openshift-kube-controller-manager/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269289934Z namespaces/openshift-kube-controller-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269357835Z namespaces/openshift-kube-scheduler-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269406296Z namespaces/openshift-kube-scheduler-operator/openshift-kube-scheduler-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269480628Z namespaces/openshift-kube-scheduler-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269515449Z namespaces/openshift-kube-scheduler-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269589521Z namespaces/openshift-kube-scheduler-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269633512Z namespaces/openshift-kube-scheduler-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269719694Z namespaces/openshift-kube-scheduler-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269801046Z namespaces/openshift-kube-scheduler-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269883408Z namespaces/openshift-kube-scheduler-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.269932989Z namespaces/openshift-kube-scheduler-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.26998036Z namespaces/openshift-kube-scheduler-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270035901Z namespaces/openshift-kube-scheduler-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270070292Z namespaces/openshift-kube-scheduler-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270272517Z namespaces/openshift-kube-scheduler-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270321868Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270363899Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270444521Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270515103Z namespaces/openshift-kube-scheduler-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270574434Z namespaces/openshift-kube-scheduler-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270677766Z namespaces/openshift-kube-scheduler-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270771199Z namespaces/openshift-kube-scheduler-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270850171Z namespaces/openshift-kube-scheduler-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.270932543Z namespaces/openshift-kube-scheduler-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271017174Z namespaces/openshift-kube-scheduler-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271155428Z namespaces/openshift-kube-scheduler-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271325762Z namespaces/openshift-kube-scheduler-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271375453Z namespaces/openshift-kube-scheduler-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271422734Z namespaces/openshift-kube-scheduler-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271468815Z namespaces/openshift-kube-scheduler-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271505476Z namespaces/openshift-kube-scheduler-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271561518Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271603238Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.27168541Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271770722Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271831164Z namespaces/openshift-kube-scheduler-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271879845Z namespaces/openshift-kube-scheduler-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271950667Z namespaces/openshift-kube-scheduler-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.271978067Z namespaces/openshift-kube-scheduler-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.272065329Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.27208302Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.27210782Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.272289484Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.272490459Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.272629353Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.272828337Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273080283Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273313939Z namespaces/openshift-kube-scheduler-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.27335927Z namespaces/openshift-kube-scheduler-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273420081Z namespaces/openshift-kube-scheduler-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273464672Z namespaces/openshift-kube-scheduler-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273526574Z namespaces/openshift-kube-scheduler/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273562895Z namespaces/openshift-kube-scheduler/openshift-kube-scheduler.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273632987Z namespaces/openshift-kube-scheduler/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273672107Z namespaces/openshift-kube-scheduler/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273710688Z namespaces/openshift-kube-scheduler/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273765399Z namespaces/openshift-kube-scheduler/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273848101Z namespaces/openshift-kube-scheduler/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.273933983Z namespaces/openshift-kube-scheduler/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274010435Z namespaces/openshift-kube-scheduler/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274054917Z namespaces/openshift-kube-scheduler/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274105598Z namespaces/openshift-kube-scheduler/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274146129Z namespaces/openshift-kube-scheduler/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.2742056Z namespaces/openshift-kube-scheduler/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274291182Z namespaces/openshift-kube-scheduler/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274347143Z namespaces/openshift-kube-scheduler/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274376054Z namespaces/openshift-kube-scheduler/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274464266Z namespaces/openshift-kube-scheduler/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274508437Z namespaces/openshift-kube-scheduler/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274557518Z namespaces/openshift-kube-scheduler/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274649311Z namespaces/openshift-kube-scheduler/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274730702Z namespaces/openshift-kube-scheduler/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274812834Z namespaces/openshift-kube-scheduler/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274903457Z namespaces/openshift-kube-scheduler/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.274980938Z namespaces/openshift-kube-scheduler/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275100611Z namespaces/openshift-kube-scheduler/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275275725Z namespaces/openshift-kube-scheduler/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275335557Z namespaces/openshift-kube-scheduler/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275371208Z namespaces/openshift-kube-scheduler/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275424349Z namespaces/openshift-kube-scheduler/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.27546664Z namespaces/openshift-kube-scheduler/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275505181Z namespaces/openshift-kube-scheduler/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275545462Z namespaces/openshift-kube-scheduler/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275625614Z namespaces/openshift-kube-scheduler/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275707586Z namespaces/openshift-kube-scheduler/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275751777Z namespaces/openshift-kube-scheduler/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275797708Z namespaces/openshift-kube-scheduler/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275847149Z namespaces/openshift-kube-scheduler/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.27588097Z namespaces/openshift-kube-scheduler/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275936141Z namespaces/openshift-kube-scheduler/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275952071Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.275992572Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.276175797Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.276358131Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.276498174Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.276700599Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.276971946Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277186611Z namespaces/openshift-kube-scheduler/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277228592Z namespaces/openshift-kube-scheduler/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277287703Z namespaces/openshift-kube-scheduler/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277321044Z namespaces/openshift-kube-scheduler/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277401776Z namespaces/openshift-kube-storage-version-migrator-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277455877Z namespaces/openshift-kube-storage-version-migrator-operator/openshift-kube-storage-version-migrator-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277550139Z namespaces/openshift-kube-storage-version-migrator-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.27758674Z namespaces/openshift-kube-storage-version-migrator-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277648502Z namespaces/openshift-kube-storage-version-migrator-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277690893Z namespaces/openshift-kube-storage-version-migrator-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277783695Z namespaces/openshift-kube-storage-version-migrator-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.277899797Z namespaces/openshift-kube-storage-version-migrator-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.27800296Z namespaces/openshift-kube-storage-version-migrator-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278042291Z namespaces/openshift-kube-storage-version-migrator-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278090622Z namespaces/openshift-kube-storage-version-migrator-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278209155Z namespaces/openshift-kube-storage-version-migrator-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278241646Z namespaces/openshift-kube-storage-version-migrator-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278333688Z namespaces/openshift-kube-storage-version-migrator-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278393519Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.27843277Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278524252Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278564003Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278572864Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278621835Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/leases/openshift-kube-storage-version-migrator-operator-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278719147Z namespaces/openshift-kube-storage-version-migrator-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278762798Z namespaces/openshift-kube-storage-version-migrator-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278891071Z namespaces/openshift-kube-storage-version-migrator-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.278970993Z namespaces/openshift-kube-storage-version-migrator-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.279190468Z namespaces/openshift-kube-storage-version-migrator-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.279295321Z namespaces/openshift-kube-storage-version-migrator-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.279426944Z namespaces/openshift-kube-storage-version-migrator-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.279556527Z namespaces/openshift-kube-storage-version-migrator-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.279738301Z namespaces/openshift-kube-storage-version-migrator-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.279801963Z namespaces/openshift-kube-storage-version-migrator-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.279837614Z namespaces/openshift-kube-storage-version-migrator-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.279896765Z namespaces/openshift-kube-storage-version-migrator-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.279934246Z namespaces/openshift-kube-storage-version-migrator-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.279993717Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.280041458Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.280165021Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.280259633Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.280315465Z namespaces/openshift-kube-storage-version-migrator-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.280385156Z namespaces/openshift-kube-storage-version-migrator-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.280439998Z namespaces/openshift-kube-storage-version-migrator-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.280462878Z namespaces/openshift-kube-storage-version-migrator-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28053194Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28054197Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.280582341Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.280747865Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28094378Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.281090553Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.281329239Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.281582395Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28179358Z namespaces/openshift-kube-storage-version-migrator-operator/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28180766Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-6769c5d45-z962g/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28181993Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-6769c5d45-z962g/kube-storage-version-migrator-operator-6769c5d45-z962g.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.281962424Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-6769c5d45-z962g/kube-storage-version-migrator-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.281971554Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-6769c5d45-z962g/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.281977764Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-6769c5d45-z962g/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.281999585Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-6769c5d45-z962g/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282151818Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-6769c5d45-z962g/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282294432Z namespaces/openshift-kube-storage-version-migrator-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282312512Z namespaces/openshift-kube-storage-version-migrator-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282389964Z namespaces/openshift-kube-storage-version-migrator-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282436185Z namespaces/openshift-kube-storage-version-migrator-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282504206Z namespaces/openshift-kube-storage-version-migrator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282542618Z namespaces/openshift-kube-storage-version-migrator/openshift-kube-storage-version-migrator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282606019Z namespaces/openshift-kube-storage-version-migrator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28264241Z namespaces/openshift-kube-storage-version-migrator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282690851Z namespaces/openshift-kube-storage-version-migrator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282744192Z namespaces/openshift-kube-storage-version-migrator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282832764Z namespaces/openshift-kube-storage-version-migrator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.282941347Z namespaces/openshift-kube-storage-version-migrator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283044329Z namespaces/openshift-kube-storage-version-migrator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28308906Z namespaces/openshift-kube-storage-version-migrator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283156482Z namespaces/openshift-kube-storage-version-migrator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283321746Z namespaces/openshift-kube-storage-version-migrator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283377147Z namespaces/openshift-kube-storage-version-migrator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28347694Z namespaces/openshift-kube-storage-version-migrator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283525661Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283563512Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283655954Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283710805Z namespaces/openshift-kube-storage-version-migrator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283749086Z namespaces/openshift-kube-storage-version-migrator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283853129Z namespaces/openshift-kube-storage-version-migrator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.283942981Z namespaces/openshift-kube-storage-version-migrator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.284072554Z namespaces/openshift-kube-storage-version-migrator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.284200667Z namespaces/openshift-kube-storage-version-migrator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28432774Z namespaces/openshift-kube-storage-version-migrator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.284460043Z namespaces/openshift-kube-storage-version-migrator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.284628507Z namespaces/openshift-kube-storage-version-migrator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.284684598Z namespaces/openshift-kube-storage-version-migrator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.284729079Z namespaces/openshift-kube-storage-version-migrator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.284783391Z namespaces/openshift-kube-storage-version-migrator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.284825542Z namespaces/openshift-kube-storage-version-migrator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.284867663Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.284903323Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.285016386Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.285095868Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28518431Z namespaces/openshift-kube-storage-version-migrator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.285251452Z namespaces/openshift-kube-storage-version-migrator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.285275742Z namespaces/openshift-kube-storage-version-migrator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.285326864Z namespaces/openshift-kube-storage-version-migrator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.285373425Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.285384015Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.285426226Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28559726Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.285777614Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.285914578Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286136613Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286389739Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286595403Z namespaces/openshift-kube-storage-version-migrator/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286602944Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286640935Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/migrator-74bb7799d9-swmml.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286726027Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/graceful-termination/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286733527Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/graceful-termination/graceful-termination/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286737117Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/graceful-termination/graceful-termination/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286786778Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/graceful-termination/graceful-termination/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28687593Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/graceful-termination/graceful-termination/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286950032Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/graceful-termination/graceful-termination/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286975403Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/migrator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286981543Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/migrator/migrator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.286996263Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/migrator/migrator/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287036754Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/migrator/migrator/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287164707Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/migrator/migrator/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287242769Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-74bb7799d9-swmml/migrator/migrator/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28728464Z namespaces/openshift-kube-storage-version-migrator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287329241Z namespaces/openshift-kube-storage-version-migrator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287373352Z namespaces/openshift-kube-storage-version-migrator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287422843Z namespaces/openshift-kube-storage-version-migrator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287481935Z namespaces/openshift-lws-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287535146Z namespaces/openshift-lws-operator/openshift-lws-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287590477Z namespaces/openshift-lws-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287636338Z namespaces/openshift-lws-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287688849Z namespaces/openshift-lws-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28772481Z namespaces/openshift-lws-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287811932Z namespaces/openshift-lws-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.287987087Z namespaces/openshift-lws-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.28810799Z namespaces/openshift-lws-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288185981Z namespaces/openshift-lws-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288232672Z namespaces/openshift-lws-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288276263Z namespaces/openshift-lws-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288326245Z namespaces/openshift-lws-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288408697Z namespaces/openshift-lws-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288465098Z namespaces/openshift-lws-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288500799Z namespaces/openshift-lws-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288595041Z namespaces/openshift-lws-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288638572Z namespaces/openshift-lws-operator/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288645842Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288693543Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/b8b2488c.x-k8s.io.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288787026Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/openshift-lws-operator-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288847307Z namespaces/openshift-lws-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.288895248Z namespaces/openshift-lws-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.289013761Z namespaces/openshift-lws-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.289099973Z namespaces/openshift-lws-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.289465742Z namespaces/openshift-lws-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.289543964Z namespaces/openshift-lws-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.289722328Z namespaces/openshift-lws-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.289865101Z namespaces/openshift-lws-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290056576Z namespaces/openshift-lws-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290142068Z namespaces/openshift-lws-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290193929Z namespaces/openshift-lws-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290271211Z namespaces/openshift-lws-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290292381Z namespaces/openshift-lws-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290358083Z namespaces/openshift-lws-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290402134Z namespaces/openshift-lws-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290483026Z namespaces/openshift-lws-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290556177Z namespaces/openshift-lws-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290612319Z namespaces/openshift-lws-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.29067254Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290719211Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290755612Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors/lws-controller-manager-metrics-monitor.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290813674Z namespaces/openshift-lws-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290853214Z namespaces/openshift-lws-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290907856Z namespaces/openshift-lws-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290917706Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.290959547Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.291140831Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.291333096Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.291492699Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/leader-worker-set.v1.0.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.291714335Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.29191726Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292150255Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292429492Z namespaces/openshift-lws-operator/operators.coreos.com/installplans/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292460502Z namespaces/openshift-lws-operator/operators.coreos.com/installplans/install-8jw8q.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292580705Z namespaces/openshift-lws-operator/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292629086Z namespaces/openshift-lws-operator/operators.coreos.com/operatorconditions/leader-worker-set.v1.0.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292692118Z namespaces/openshift-lws-operator/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292730619Z namespaces/openshift-lws-operator/operators.coreos.com/operatorgroups/leader-worker-set.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.29278749Z namespaces/openshift-lws-operator/operators.coreos.com/subscriptions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292859882Z namespaces/openshift-lws-operator/operators.coreos.com/subscriptions/leader-worker-set.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292941814Z namespaces/openshift-lws-operator/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292949384Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-8xtsj/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.292993615Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-8xtsj/lws-controller-manager-7f997f587c-8xtsj.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293089957Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-8xtsj/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293096808Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-8xtsj/manager/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293100588Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-8xtsj/manager/manager/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293174619Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-8xtsj/manager/manager/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293299592Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-8xtsj/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293362434Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-8xtsj/manager/manager/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293415385Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-sbdmj/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293446336Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-sbdmj/lws-controller-manager-7f997f587c-sbdmj.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293540308Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-sbdmj/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293551768Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-sbdmj/manager/manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293556208Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-sbdmj/manager/manager/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293584809Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-sbdmj/manager/manager/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293740943Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-sbdmj/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293812114Z namespaces/openshift-lws-operator/pods/lws-controller-manager-7f997f587c-sbdmj/manager/manager/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293846825Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-bfc7f696d-w7ggm/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293897747Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-bfc7f696d-w7ggm/openshift-lws-operator-bfc7f696d-w7ggm.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293978738Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-bfc7f696d-w7ggm/openshift-lws-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293987349Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-bfc7f696d-w7ggm/openshift-lws-operator/openshift-lws-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.293991859Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-bfc7f696d-w7ggm/openshift-lws-operator/openshift-lws-operator/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.29402645Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-bfc7f696d-w7ggm/openshift-lws-operator/openshift-lws-operator/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294222894Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-bfc7f696d-w7ggm/openshift-lws-operator/openshift-lws-operator/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294297446Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-bfc7f696d-w7ggm/openshift-lws-operator/openshift-lws-operator/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294319517Z namespaces/openshift-lws-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294370708Z namespaces/openshift-lws-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294412409Z namespaces/openshift-lws-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.29446439Z namespaces/openshift-lws-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294510421Z namespaces/openshift-machine-api/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294520131Z namespaces/openshift-machine-api/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294528091Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294575413Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294728486Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.294911711Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.295052784Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.295287969Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.295533905Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.29574504Z namespaces/openshift-machine-config-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.295757591Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.295764311Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.295793201Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/machine-config-controller.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.295900314Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/machine-config-daemon.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.295973876Z namespaces/openshift-machine-config-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.295982106Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.296022657Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.296206261Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.296396816Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.296540899Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.296746454Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.29700102Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297238976Z namespaces/openshift-marketplace/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297250346Z namespaces/openshift-marketplace/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297254046Z namespaces/openshift-marketplace/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297281517Z namespaces/openshift-marketplace/monitoring.coreos.com/prometheusrules/marketplace-alert-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297351269Z namespaces/openshift-marketplace/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297358469Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.29741211Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/certified-operators.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297504562Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/community-operators.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297587174Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/redhat-marketplace.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297680816Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/redhat-operators.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297731087Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297763528Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.297929502Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.298108926Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.298308231Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.298512476Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.298758452Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.298994237Z namespaces/openshift-monitoring/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.299033308Z namespaces/openshift-monitoring/openshift-monitoring.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.29911693Z namespaces/openshift-monitoring/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.299177322Z namespaces/openshift-monitoring/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.299234843Z namespaces/openshift-monitoring/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.299265894Z namespaces/openshift-monitoring/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.299422427Z namespaces/openshift-monitoring/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.299863898Z namespaces/openshift-monitoring/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.300264787Z namespaces/openshift-monitoring/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.300486403Z namespaces/openshift-monitoring/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.300562985Z namespaces/openshift-monitoring/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.300616736Z namespaces/openshift-monitoring/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.300662507Z namespaces/openshift-monitoring/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.300743099Z namespaces/openshift-monitoring/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.30079974Z namespaces/openshift-monitoring/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.300836691Z namespaces/openshift-monitoring/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.300932663Z namespaces/openshift-monitoring/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.300993455Z namespaces/openshift-monitoring/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.301030816Z namespaces/openshift-monitoring/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.30417981Z namespaces/openshift-monitoring/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.304342934Z namespaces/openshift-monitoring/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.305407389Z namespaces/openshift-monitoring/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.305474761Z namespaces/openshift-monitoring/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.306447724Z namespaces/openshift-monitoring/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.306582477Z namespaces/openshift-monitoring/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.307070609Z namespaces/openshift-monitoring/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.307259423Z namespaces/openshift-monitoring/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.307305574Z namespaces/openshift-monitoring/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.307455118Z namespaces/openshift-monitoring/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.307493759Z namespaces/openshift-monitoring/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.30754383Z namespaces/openshift-monitoring/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.307586211Z namespaces/openshift-monitoring/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.307682993Z namespaces/openshift-monitoring/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.307763835Z namespaces/openshift-monitoring/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.307816196Z namespaces/openshift-monitoring/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.307849637Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.308007601Z namespaces/openshift-monitoring/monitoring.coreos.com/alertmanagers/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.308038922Z namespaces/openshift-monitoring/monitoring.coreos.com/alertmanagers/main.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.308158574Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheuses/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.308208056Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheuses/k8s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.308295608Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.308325389Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/alertmanager-main-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.308438171Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/cluster-monitoring-operator-prometheus-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.308628836Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/kube-state-metrics-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.308715838Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/kubernetes-monitoring-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.308985814Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/node-exporter-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.309118327Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-k8s-prometheus-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.309266221Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-k8s-thanos-sidecar-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.309360823Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-operator-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.309474756Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/telemetry.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.309566358Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/thanos-querier.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.309640369Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.309691291Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/alertmanager-main.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.309778093Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/cluster-monitoring-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.309860165Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/kube-state-metrics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.309952967Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/kubelet.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.310053509Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/metrics-server.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.310148752Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/node-exporter.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.310326836Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/openshift-state-metrics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.310418408Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/prometheus-k8s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3105068Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/prometheus-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.310600262Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/telemeter-client.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.310688004Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/thanos-querier.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.310776736Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/thanos-sidecar.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.310836988Z namespaces/openshift-monitoring/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.310877159Z namespaces/openshift-monitoring/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31093097Z namespaces/openshift-monitoring/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31094207Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.310978931Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.311186876Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.311423752Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.311576435Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.311790851Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312051987Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312351484Z namespaces/openshift-monitoring/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312381174Z namespaces/openshift-monitoring/operators.coreos.com/operatorgroups/openshift-cluster-monitoring.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312470127Z namespaces/openshift-monitoring/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312480367Z namespaces/openshift-monitoring/pods/alertmanager-main-0/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312561029Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager-main-0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312724153Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312735853Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312739773Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312787114Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312880646Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312959108Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312992229Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.312998809Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313010839Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31305177Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313172113Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313242495Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313272586Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313278736Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313285126Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313326667Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313423159Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313493891Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313532032Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313540602Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313544592Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313582733Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313675465Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313742637Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313778587Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313795248Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313800178Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313834279Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313928841Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.313994793Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314022333Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314029624Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314036914Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314074865Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314238838Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31431225Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314341701Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314349951Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314353701Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314410213Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314496564Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314562566Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314606187Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-75587bd455-lhgp5/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314643608Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-75587bd455-lhgp5/cluster-monitoring-operator-75587bd455-lhgp5.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31473592Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-75587bd455-lhgp5/cluster-monitoring-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31474363Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-75587bd455-lhgp5/cluster-monitoring-operator/cluster-monitoring-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314747621Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-75587bd455-lhgp5/cluster-monitoring-operator/cluster-monitoring-operator/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.314791171Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-75587bd455-lhgp5/cluster-monitoring-operator/cluster-monitoring-operator/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315309464Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-75587bd455-lhgp5/cluster-monitoring-operator/cluster-monitoring-operator/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315388166Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-75587bd455-lhgp5/cluster-monitoring-operator/cluster-monitoring-operator/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315433277Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315488198Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-state-metrics-69db897b98-x7rd5.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31558179Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-main/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315588661Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-main/kube-rbac-proxy-main/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315592301Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315639442Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315740494Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315810706Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315841376Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-self/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315849997Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-self/kube-rbac-proxy-self/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315853627Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.315908118Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31600422Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316075572Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316136154Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-state-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316147394Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-state-metrics/kube-state-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316157224Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-state-metrics/kube-state-metrics/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316197355Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-state-metrics/kube-state-metrics/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316336728Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-state-metrics/kube-state-metrics/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31640604Z namespaces/openshift-monitoring/pods/kube-state-metrics-69db897b98-x7rd5/kube-state-metrics/kube-state-metrics/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316448311Z namespaces/openshift-monitoring/pods/metrics-server-6fd764c9f6-wvdt6/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316492662Z namespaces/openshift-monitoring/pods/metrics-server-6fd764c9f6-wvdt6/metrics-server-6fd764c9f6-wvdt6.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316583234Z namespaces/openshift-monitoring/pods/metrics-server-6fd764c9f6-wvdt6/metrics-server/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316590614Z namespaces/openshift-monitoring/pods/metrics-server-6fd764c9f6-wvdt6/metrics-server/metrics-server/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316594134Z namespaces/openshift-monitoring/pods/metrics-server-6fd764c9f6-wvdt6/metrics-server/metrics-server/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316632825Z namespaces/openshift-monitoring/pods/metrics-server-6fd764c9f6-wvdt6/metrics-server/metrics-server/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316727048Z namespaces/openshift-monitoring/pods/metrics-server-6fd764c9f6-wvdt6/metrics-server/metrics-server/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316793409Z namespaces/openshift-monitoring/pods/metrics-server-6fd764c9f6-wvdt6/metrics-server/metrics-server/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3168437Z namespaces/openshift-monitoring/pods/monitoring-plugin-7dccd58f55-6hk4n/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316885711Z namespaces/openshift-monitoring/pods/monitoring-plugin-7dccd58f55-6hk4n/monitoring-plugin-7dccd58f55-6hk4n.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316972173Z namespaces/openshift-monitoring/pods/monitoring-plugin-7dccd58f55-6hk4n/monitoring-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316979594Z namespaces/openshift-monitoring/pods/monitoring-plugin-7dccd58f55-6hk4n/monitoring-plugin/monitoring-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.316983223Z namespaces/openshift-monitoring/pods/monitoring-plugin-7dccd58f55-6hk4n/monitoring-plugin/monitoring-plugin/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317020044Z namespaces/openshift-monitoring/pods/monitoring-plugin-7dccd58f55-6hk4n/monitoring-plugin/monitoring-plugin/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317139807Z namespaces/openshift-monitoring/pods/monitoring-plugin-7dccd58f55-6hk4n/monitoring-plugin/monitoring-plugin/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317210979Z namespaces/openshift-monitoring/pods/monitoring-plugin-7dccd58f55-6hk4n/monitoring-plugin/monitoring-plugin/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31725306Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317288551Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/node-exporter-dpzm9.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317401273Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/init-textfile/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317408784Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/init-textfile/init-textfile/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317415034Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/init-textfile/init-textfile/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317463015Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317542427Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317618218Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317661339Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31766843Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31767795Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317721241Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317806903Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317879175Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317915826Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/node-exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317924526Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/node-exporter/node-exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317934246Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/node-exporter/node-exporter/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.317979527Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31808388Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318179112Z namespaces/openshift-monitoring/pods/node-exporter-dpzm9/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318213903Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318263274Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/node-exporter-grgjm.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318361056Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/init-textfile/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318368456Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/init-textfile/init-textfile/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318371896Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/init-textfile/init-textfile/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318414868Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318494609Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318564291Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318599802Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318608812Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318614662Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318654423Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318741845Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318812897Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318857208Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/node-exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318870598Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/node-exporter/node-exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.318876448Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/node-exporter/node-exporter/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31892456Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319029872Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319097104Z namespaces/openshift-monitoring/pods/node-exporter-grgjm/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319165245Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319210886Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/node-exporter-nk67g.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319302358Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/init-textfile/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319309159Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/init-textfile/init-textfile/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319318069Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/init-textfile/init-textfile/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31936225Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319436122Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319499343Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319534454Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319541474Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319544864Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319585855Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319677847Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319751179Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31978734Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/node-exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.31979753Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/node-exporter/node-exporter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319807281Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/node-exporter/node-exporter/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319852702Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.319970774Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320034576Z namespaces/openshift-monitoring/pods/node-exporter-nk67g/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320094657Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320164879Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/openshift-state-metrics-9d44df66c-qpm5s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320301622Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-main/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320308922Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-main/kube-rbac-proxy-main/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320316393Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320369434Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320465256Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320543068Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320583659Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-self/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320592489Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-self/kube-rbac-proxy-self/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320610909Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3206519Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320744483Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320818734Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320863645Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/openshift-state-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320872716Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/openshift-state-metrics/openshift-state-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320876936Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/openshift-state-metrics/openshift-state-metrics/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.320932907Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/openshift-state-metrics/openshift-state-metrics/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32103471Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/openshift-state-metrics/openshift-state-metrics/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321106981Z namespaces/openshift-monitoring/pods/openshift-state-metrics-9d44df66c-qpm5s/openshift-state-metrics/openshift-state-metrics/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321173683Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321245314Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus-k8s-0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321417608Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321425649Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321429219Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32147368Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321575932Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321644134Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321682725Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321688955Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321692395Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321740896Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321832738Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3218982Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321939891Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321948141Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.321955481Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322007242Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322096055Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322200787Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322237658Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322246058Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322249918Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322301389Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322385722Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322462303Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322497704Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322504484Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322510204Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322550135Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322645597Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322715019Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32275077Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32275759Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322772771Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.322808552Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323373855Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323447957Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323474057Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323482288Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323486048Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323533019Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323632661Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323699973Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323739154Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323776324Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/prometheus-operator-5676c8c784-v7997.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323863426Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323871817Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323875977Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.323903598Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32400008Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324077171Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324093862Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/prometheus-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324103122Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/prometheus-operator/prometheus-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324112412Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/prometheus-operator/prometheus-operator/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324187224Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/prometheus-operator/prometheus-operator/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324401549Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/prometheus-operator/prometheus-operator/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324475511Z namespaces/openshift-monitoring/pods/prometheus-operator-5676c8c784-v7997/prometheus-operator/prometheus-operator/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324510982Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-57cf98b594-fthv5/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324553283Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-57cf98b594-fthv5/prometheus-operator-admission-webhook-57cf98b594-fthv5.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324679836Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-57cf98b594-fthv5/prometheus-operator-admission-webhook/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324688326Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-57cf98b594-fthv5/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324692576Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-57cf98b594-fthv5/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324729177Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-57cf98b594-fthv5/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32483386Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-57cf98b594-fthv5/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324904951Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-57cf98b594-fthv5/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.324965192Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325012714Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/telemeter-client-68d84d7d56-rb9td.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325193738Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325205728Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325209788Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325218759Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325321741Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325401653Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325418583Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/reload/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325426754Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/reload/reload/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325430684Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/reload/reload/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325478785Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/reload/reload/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325578817Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/reload/reload/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325647119Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/reload/reload/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32568421Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/telemeter-client/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3256916Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/telemeter-client/telemeter-client/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32569745Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/telemeter-client/telemeter-client/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325743341Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/telemeter-client/telemeter-client/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325889105Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/telemeter-client/telemeter-client/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325957296Z namespaces/openshift-monitoring/pods/telemeter-client-68d84d7d56-rb9td/telemeter-client/telemeter-client/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.325996847Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326062089Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/thanos-querier-7d86fc6796-gc8qh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326217502Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326232593Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326236553Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326282754Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326369676Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326437958Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326488299Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-rules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326496459Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-rules/kube-rbac-proxy-rules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326509829Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32654868Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326635892Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326702314Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326748275Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-web/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326759465Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326763675Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326788446Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326879638Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32695112Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326987371Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326993901Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.326997571Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327028532Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327135134Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327214426Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327290908Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/prom-label-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327302928Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/prom-label-proxy/prom-label-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327309168Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/prom-label-proxy/prom-label-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327347459Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/prom-label-proxy/prom-label-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327433641Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/prom-label-proxy/prom-label-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327509613Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/prom-label-proxy/prom-label-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327526303Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/thanos-query/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327551464Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/thanos-query/thanos-query/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327562334Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/thanos-query/thanos-query/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327608575Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/thanos-query/thanos-query/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327710978Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/thanos-query/thanos-query/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327780609Z namespaces/openshift-monitoring/pods/thanos-querier-7d86fc6796-gc8qh/thanos-query/thanos-query/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.32781086Z namespaces/openshift-monitoring/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327861091Z namespaces/openshift-monitoring/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327910813Z namespaces/openshift-monitoring/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.327959663Z namespaces/openshift-monitoring/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328048466Z namespaces/openshift-multus/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328085057Z namespaces/openshift-multus/openshift-multus.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328177289Z namespaces/openshift-multus/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328198619Z namespaces/openshift-multus/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328278201Z namespaces/openshift-multus/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328295742Z namespaces/openshift-multus/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328523967Z namespaces/openshift-multus/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328605399Z namespaces/openshift-multus/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328680141Z namespaces/openshift-multus/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328726842Z namespaces/openshift-multus/apps/daemonsets/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328775163Z namespaces/openshift-multus/apps/daemonsets/multus-additional-cni-plugins.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.328935997Z namespaces/openshift-multus/apps/daemonsets/multus.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329056349Z namespaces/openshift-multus/apps/daemonsets/network-metrics-daemon.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329110871Z namespaces/openshift-multus/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329183812Z namespaces/openshift-multus/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329235264Z namespaces/openshift-multus/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329279505Z namespaces/openshift-multus/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329358297Z namespaces/openshift-multus/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329409738Z namespaces/openshift-multus/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329440639Z namespaces/openshift-multus/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329529921Z namespaces/openshift-multus/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329596242Z namespaces/openshift-multus/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329651404Z namespaces/openshift-multus/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329786667Z namespaces/openshift-multus/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.329872989Z namespaces/openshift-multus/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.330434552Z namespaces/openshift-multus/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.331286412Z namespaces/openshift-multus/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.331866556Z namespaces/openshift-multus/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.331954658Z namespaces/openshift-multus/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332091301Z namespaces/openshift-multus/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332165003Z namespaces/openshift-multus/core/configmaps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332213534Z namespaces/openshift-multus/core/configmaps/cni-copy-resources.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332309937Z namespaces/openshift-multus/core/configmaps/default-cni-sysctl-allowlist.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332401799Z namespaces/openshift-multus/core/configmaps/multus-daemon-config.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332491221Z namespaces/openshift-multus/core/configmaps/whereabouts-flatfile-config.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332576933Z namespaces/openshift-multus/core/serviceaccounts/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332619764Z namespaces/openshift-multus/core/serviceaccounts/metrics-daemon-sa.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332703096Z namespaces/openshift-multus/core/serviceaccounts/multus-ac.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332791538Z namespaces/openshift-multus/core/serviceaccounts/multus-ancillary-tools.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.33287035Z namespaces/openshift-multus/core/serviceaccounts/multus.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332912121Z namespaces/openshift-multus/core/services/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.332969542Z namespaces/openshift-multus/core/services/network-metrics-service.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333026974Z namespaces/openshift-multus/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333067464Z namespaces/openshift-multus/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333138066Z namespaces/openshift-multus/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333176037Z namespaces/openshift-multus/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333236939Z namespaces/openshift-multus/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333269429Z namespaces/openshift-multus/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333360331Z namespaces/openshift-multus/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333439503Z namespaces/openshift-multus/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333481384Z namespaces/openshift-multus/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333529546Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333587667Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333640738Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors/monitor-network.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333689289Z namespaces/openshift-multus/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.33373962Z namespaces/openshift-multus/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333797792Z namespaces/openshift-multus/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333808662Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333846023Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.333993786Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.334191001Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.334332905Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.33454982Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.334802746Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.33500247Z namespaces/openshift-multus/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.335011651Z namespaces/openshift-multus/pods/multus-9rv78/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.335047402Z namespaces/openshift-multus/pods/multus-9rv78/multus-9rv78.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.335172514Z namespaces/openshift-multus/pods/multus-9rv78/kube-multus/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.335187095Z namespaces/openshift-multus/pods/multus-9rv78/kube-multus/kube-multus/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.335191575Z namespaces/openshift-multus/pods/multus-9rv78/kube-multus/kube-multus/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.335220846Z namespaces/openshift-multus/pods/multus-9rv78/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.335784729Z namespaces/openshift-multus/pods/multus-9rv78/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.335854981Z namespaces/openshift-multus/pods/multus-9rv78/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.335893111Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.335942403Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/multus-additional-cni-plugins-5shvq.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336084786Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/bond-cni-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336094036Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336098566Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336155668Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.33626218Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336328792Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336353642Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336358393Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/cni-plugins/cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336372723Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336417724Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336513876Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336581928Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336621969Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/egress-router-binary-copy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336628709Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336638689Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.33669355Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336794983Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336867675Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336908326Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336925996Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336937586Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.336980397Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337053479Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337136031Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337193672Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/routeoverride-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337203182Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337208923Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337247604Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337339206Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337419638Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337447428Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni-bincopy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337453728Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337461519Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337487649Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337578302Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337652283Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337689944Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337697324Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337735005Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337780766Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337868548Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3379436Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5shvq/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.337982601Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338040202Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/multus-additional-cni-plugins-kqxwj.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338254938Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/bond-cni-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338268038Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338273958Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338286988Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338395261Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338466363Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338501653Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338511413Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/cni-plugins/cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338517384Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338537164Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338633767Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338701968Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338739759Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/egress-router-binary-copy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338750649Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338754539Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.33878803Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338878332Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.338987575Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339025576Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339034216Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339037816Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339107618Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.33921629Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339307852Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339337063Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/routeoverride-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339348033Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339352184Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339387504Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339478147Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339547308Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339582289Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni-bincopy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339589909Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339593529Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.33963791Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339731712Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339800644Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339834585Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339842205Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339845865Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339889336Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.339975288Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34005866Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-kqxwj/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340074421Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340155012Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/multus-additional-cni-plugins-ppnfd.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340298226Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/bond-cni-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340310216Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340317436Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340343317Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340441089Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340514051Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340538901Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340547662Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/cni-plugins/cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340551982Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340605393Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340698325Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340774037Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340787927Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/egress-router-binary-copy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340796078Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340812858Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340853759Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.340962111Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341036873Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341074124Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341081974Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341085485Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341115875Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341221488Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341291139Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34133578Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/routeoverride-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341351851Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341358411Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341407872Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341500804Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341574366Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341610667Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni-bincopy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341620297Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341624067Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341654638Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3417467Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341815272Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341856993Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341868883Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341874973Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341901584Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.341991956Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.342062868Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ppnfd/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.342100869Z namespaces/openshift-multus/pods/multus-df2mh/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34216056Z namespaces/openshift-multus/pods/multus-df2mh/multus-df2mh.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.342264992Z namespaces/openshift-multus/pods/multus-df2mh/kube-multus/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.342273363Z namespaces/openshift-multus/pods/multus-df2mh/kube-multus/kube-multus/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.342277403Z namespaces/openshift-multus/pods/multus-df2mh/kube-multus/kube-multus/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.342308914Z namespaces/openshift-multus/pods/multus-df2mh/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.342886537Z namespaces/openshift-multus/pods/multus-df2mh/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.342954069Z namespaces/openshift-multus/pods/multus-df2mh/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34298899Z namespaces/openshift-multus/pods/multus-f5cf5/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34301949Z namespaces/openshift-multus/pods/multus-f5cf5/multus-f5cf5.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343114582Z namespaces/openshift-multus/pods/multus-f5cf5/kube-multus/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343141713Z namespaces/openshift-multus/pods/multus-f5cf5/kube-multus/kube-multus/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343148903Z namespaces/openshift-multus/pods/multus-f5cf5/kube-multus/kube-multus/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343189344Z namespaces/openshift-multus/pods/multus-f5cf5/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343570883Z namespaces/openshift-multus/pods/multus-f5cf5/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343642825Z namespaces/openshift-multus/pods/multus-f5cf5/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343673836Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343703196Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/network-metrics-daemon-ldwdd.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343794659Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343802099Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343805959Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34383819Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.343965863Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344036584Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344073425Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/network-metrics-daemon/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344082436Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344086416Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344132707Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344300881Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344369882Z namespaces/openshift-multus/pods/network-metrics-daemon-ldwdd/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344401653Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344452634Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/network-metrics-daemon-n487x.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344518376Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344527016Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344533106Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344566747Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344654239Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344721931Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344758921Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/network-metrics-daemon/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344773672Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344777342Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344814963Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.344974637Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345042808Z namespaces/openshift-multus/pods/network-metrics-daemon-n487x/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345086559Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345146151Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/network-metrics-daemon-v4zl5.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345225823Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345232933Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345237183Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345276364Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345371686Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345444998Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345473889Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/network-metrics-daemon/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345480039Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345484139Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34553245Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345674393Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345744715Z namespaces/openshift-multus/pods/network-metrics-daemon-v4zl5/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345776666Z namespaces/openshift-multus/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345807856Z namespaces/openshift-multus/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345863638Z namespaces/openshift-multus/rbac.authorization.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345871608Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.345913499Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/multus-whereabouts.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346001001Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346056192Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346092603Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346252757Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/whereabouts-cni.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346314578Z namespaces/openshift-multus/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346352019Z namespaces/openshift-multus/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34639174Z namespaces/openshift-must-gather-xhf46/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346399651Z namespaces/openshift-must-gather-xhf46/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346403491Z namespaces/openshift-must-gather-xhf46/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346445181Z namespaces/openshift-must-gather-xhf46/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346627106Z namespaces/openshift-must-gather-xhf46/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34682547Z namespaces/openshift-must-gather-xhf46/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.346958774Z namespaces/openshift-must-gather-xhf46/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.347173899Z namespaces/openshift-must-gather-xhf46/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.347415335Z namespaces/openshift-must-gather-xhf46/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34762438Z namespaces/openshift-network-console/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34765343Z namespaces/openshift-network-console/openshift-network-console.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.347726882Z namespaces/openshift-network-console/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.347759393Z namespaces/openshift-network-console/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.347817134Z namespaces/openshift-network-console/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.347856235Z namespaces/openshift-network-console/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.347939137Z namespaces/openshift-network-console/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34805728Z namespaces/openshift-network-console/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348181353Z namespaces/openshift-network-console/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348211233Z namespaces/openshift-network-console/apps/deployments/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348265325Z namespaces/openshift-network-console/apps/deployments/networking-console-plugin.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348343066Z namespaces/openshift-network-console/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348389798Z namespaces/openshift-network-console/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348420658Z namespaces/openshift-network-console/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348456939Z namespaces/openshift-network-console/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348541731Z namespaces/openshift-network-console/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348578002Z namespaces/openshift-network-console/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348629183Z namespaces/openshift-network-console/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348705345Z namespaces/openshift-network-console/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348751086Z namespaces/openshift-network-console/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348790237Z namespaces/openshift-network-console/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.34889756Z namespaces/openshift-network-console/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.348995602Z namespaces/openshift-network-console/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349114805Z namespaces/openshift-network-console/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349244898Z namespaces/openshift-network-console/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349386361Z namespaces/openshift-network-console/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349507674Z namespaces/openshift-network-console/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349626907Z namespaces/openshift-network-console/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349682308Z namespaces/openshift-network-console/core/configmaps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349723519Z namespaces/openshift-network-console/core/configmaps/networking-console-plugin.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349788811Z namespaces/openshift-network-console/core/services/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349829872Z namespaces/openshift-network-console/core/services/networking-console-plugin.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349901003Z namespaces/openshift-network-console/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349944464Z namespaces/openshift-network-console/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.349995856Z namespaces/openshift-network-console/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350040586Z namespaces/openshift-network-console/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350089418Z namespaces/openshift-network-console/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350155069Z namespaces/openshift-network-console/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350231601Z namespaces/openshift-network-console/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350315563Z namespaces/openshift-network-console/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350353524Z namespaces/openshift-network-console/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350399055Z namespaces/openshift-network-console/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350446816Z namespaces/openshift-network-console/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350494717Z namespaces/openshift-network-console/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350537178Z namespaces/openshift-network-console/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350545029Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350581229Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350733473Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.350919167Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.351058091Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.351273466Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.351510092Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.351697466Z namespaces/openshift-network-console/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.351705576Z namespaces/openshift-network-console/pods/networking-console-plugin-cb95c66f6-499sv/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.351728647Z namespaces/openshift-network-console/pods/networking-console-plugin-cb95c66f6-499sv/networking-console-plugin-cb95c66f6-499sv.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.351825939Z namespaces/openshift-network-console/pods/networking-console-plugin-cb95c66f6-499sv/networking-console-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.351835069Z namespaces/openshift-network-console/pods/networking-console-plugin-cb95c66f6-499sv/networking-console-plugin/networking-console-plugin/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.351838649Z namespaces/openshift-network-console/pods/networking-console-plugin-cb95c66f6-499sv/networking-console-plugin/networking-console-plugin/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.35187228Z namespaces/openshift-network-console/pods/networking-console-plugin-cb95c66f6-499sv/networking-console-plugin/networking-console-plugin/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.351978472Z namespaces/openshift-network-console/pods/networking-console-plugin-cb95c66f6-499sv/networking-console-plugin/networking-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352042724Z namespaces/openshift-network-console/pods/networking-console-plugin-cb95c66f6-499sv/networking-console-plugin/networking-console-plugin/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352074445Z namespaces/openshift-network-console/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352146467Z namespaces/openshift-network-console/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352188648Z namespaces/openshift-network-console/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352217008Z namespaces/openshift-network-console/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352269659Z namespaces/openshift-network-diagnostics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352329931Z namespaces/openshift-network-diagnostics/openshift-network-diagnostics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352375812Z namespaces/openshift-network-diagnostics/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352427273Z namespaces/openshift-network-diagnostics/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352472144Z namespaces/openshift-network-diagnostics/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352511235Z namespaces/openshift-network-diagnostics/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352613338Z namespaces/openshift-network-diagnostics/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352712Z namespaces/openshift-network-diagnostics/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352809212Z namespaces/openshift-network-diagnostics/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352848193Z namespaces/openshift-network-diagnostics/apps/daemonsets/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352890734Z namespaces/openshift-network-diagnostics/apps/daemonsets/network-check-target.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.352966546Z namespaces/openshift-network-diagnostics/apps/deployments/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353001877Z namespaces/openshift-network-diagnostics/apps/deployments/network-check-source.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353073038Z namespaces/openshift-network-diagnostics/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353115359Z namespaces/openshift-network-diagnostics/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353186731Z namespaces/openshift-network-diagnostics/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353234032Z namespaces/openshift-network-diagnostics/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353311094Z namespaces/openshift-network-diagnostics/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353361515Z namespaces/openshift-network-diagnostics/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353403946Z namespaces/openshift-network-diagnostics/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353487928Z namespaces/openshift-network-diagnostics/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353532189Z namespaces/openshift-network-diagnostics/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353578381Z namespaces/openshift-network-diagnostics/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353675403Z namespaces/openshift-network-diagnostics/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353760845Z namespaces/openshift-network-diagnostics/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.353954439Z namespaces/openshift-network-diagnostics/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354032951Z namespaces/openshift-network-diagnostics/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354314398Z namespaces/openshift-network-diagnostics/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.35440962Z namespaces/openshift-network-diagnostics/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354514563Z namespaces/openshift-network-diagnostics/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354571444Z namespaces/openshift-network-diagnostics/core/serviceaccounts/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354617305Z namespaces/openshift-network-diagnostics/core/serviceaccounts/network-diagnostics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354660226Z namespaces/openshift-network-diagnostics/core/services/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354708447Z namespaces/openshift-network-diagnostics/core/services/network-check-source.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354787239Z namespaces/openshift-network-diagnostics/core/services/network-check-target.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.35483834Z namespaces/openshift-network-diagnostics/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354884312Z namespaces/openshift-network-diagnostics/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354937143Z namespaces/openshift-network-diagnostics/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.354979094Z namespaces/openshift-network-diagnostics/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355031625Z namespaces/openshift-network-diagnostics/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355076016Z namespaces/openshift-network-diagnostics/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355182688Z namespaces/openshift-network-diagnostics/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.35525964Z namespaces/openshift-network-diagnostics/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355308751Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355349983Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355405764Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355456495Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors/network-check-source.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355506136Z namespaces/openshift-network-diagnostics/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355540407Z namespaces/openshift-network-diagnostics/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355594448Z namespaces/openshift-network-diagnostics/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355603838Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355638379Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355783073Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.355959727Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.35609929Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.356322255Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.356561761Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.356762806Z namespaces/openshift-network-diagnostics/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.356771616Z namespaces/openshift-network-diagnostics/pods/network-check-source-8894fc9bd-kc9q6/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.356803307Z namespaces/openshift-network-diagnostics/pods/network-check-source-8894fc9bd-kc9q6/network-check-source-8894fc9bd-kc9q6.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.356883299Z namespaces/openshift-network-diagnostics/pods/network-check-source-8894fc9bd-kc9q6/check-endpoints/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.356891269Z namespaces/openshift-network-diagnostics/pods/network-check-source-8894fc9bd-kc9q6/check-endpoints/check-endpoints/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.356898409Z namespaces/openshift-network-diagnostics/pods/network-check-source-8894fc9bd-kc9q6/check-endpoints/check-endpoints/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.356951701Z namespaces/openshift-network-diagnostics/pods/network-check-source-8894fc9bd-kc9q6/check-endpoints/check-endpoints/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357063783Z namespaces/openshift-network-diagnostics/pods/network-check-source-8894fc9bd-kc9q6/check-endpoints/check-endpoints/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357154485Z namespaces/openshift-network-diagnostics/pods/network-check-source-8894fc9bd-kc9q6/check-endpoints/check-endpoints/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357173326Z namespaces/openshift-network-diagnostics/pods/network-check-target-5shjj/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357228247Z namespaces/openshift-network-diagnostics/pods/network-check-target-5shjj/network-check-target-5shjj.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357305719Z namespaces/openshift-network-diagnostics/pods/network-check-target-5shjj/network-check-target-container/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357317119Z namespaces/openshift-network-diagnostics/pods/network-check-target-5shjj/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357322029Z namespaces/openshift-network-diagnostics/pods/network-check-target-5shjj/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3573548Z namespaces/openshift-network-diagnostics/pods/network-check-target-5shjj/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357449112Z namespaces/openshift-network-diagnostics/pods/network-check-target-5shjj/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357508264Z namespaces/openshift-network-diagnostics/pods/network-check-target-5shjj/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357556845Z namespaces/openshift-network-diagnostics/pods/network-check-target-76jvz/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357592346Z namespaces/openshift-network-diagnostics/pods/network-check-target-76jvz/network-check-target-76jvz.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357667827Z namespaces/openshift-network-diagnostics/pods/network-check-target-76jvz/network-check-target-container/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357675018Z namespaces/openshift-network-diagnostics/pods/network-check-target-76jvz/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357678958Z namespaces/openshift-network-diagnostics/pods/network-check-target-76jvz/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357706538Z namespaces/openshift-network-diagnostics/pods/network-check-target-76jvz/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.35780037Z namespaces/openshift-network-diagnostics/pods/network-check-target-76jvz/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357873082Z namespaces/openshift-network-diagnostics/pods/network-check-target-76jvz/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357901003Z namespaces/openshift-network-diagnostics/pods/network-check-target-qbhf5/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.357947684Z namespaces/openshift-network-diagnostics/pods/network-check-target-qbhf5/network-check-target-qbhf5.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358018786Z namespaces/openshift-network-diagnostics/pods/network-check-target-qbhf5/network-check-target-container/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358030776Z namespaces/openshift-network-diagnostics/pods/network-check-target-qbhf5/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358034406Z namespaces/openshift-network-diagnostics/pods/network-check-target-qbhf5/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358063717Z namespaces/openshift-network-diagnostics/pods/network-check-target-qbhf5/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358178119Z namespaces/openshift-network-diagnostics/pods/network-check-target-qbhf5/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358245821Z namespaces/openshift-network-diagnostics/pods/network-check-target-qbhf5/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358282512Z namespaces/openshift-network-diagnostics/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358328723Z namespaces/openshift-network-diagnostics/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358377654Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358385054Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358418725Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/network-diagnostics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358504297Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358553428Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358598479Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/network-diagnostics.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358678341Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358728183Z namespaces/openshift-network-diagnostics/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358765743Z namespaces/openshift-network-diagnostics/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358818585Z namespaces/openshift-network-node-identity/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358870376Z namespaces/openshift-network-node-identity/openshift-network-node-identity.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358927047Z namespaces/openshift-network-node-identity/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.358970868Z namespaces/openshift-network-node-identity/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359014629Z namespaces/openshift-network-node-identity/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3590557Z namespaces/openshift-network-node-identity/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359178333Z namespaces/openshift-network-node-identity/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359308816Z namespaces/openshift-network-node-identity/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359386868Z namespaces/openshift-network-node-identity/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359440089Z namespaces/openshift-network-node-identity/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.35949499Z namespaces/openshift-network-node-identity/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359524131Z namespaces/openshift-network-node-identity/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359573952Z namespaces/openshift-network-node-identity/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359650044Z namespaces/openshift-network-node-identity/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359687595Z namespaces/openshift-network-node-identity/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359724526Z namespaces/openshift-network-node-identity/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359806278Z namespaces/openshift-network-node-identity/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359856269Z namespaces/openshift-network-node-identity/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359864799Z namespaces/openshift-network-node-identity/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.35989337Z namespaces/openshift-network-node-identity/coordination.k8s.io/leases/ovnkube-identity.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359953101Z namespaces/openshift-network-node-identity/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.359996642Z namespaces/openshift-network-node-identity/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360093455Z namespaces/openshift-network-node-identity/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360197267Z namespaces/openshift-network-node-identity/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360280709Z namespaces/openshift-network-node-identity/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360359341Z namespaces/openshift-network-node-identity/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360441413Z namespaces/openshift-network-node-identity/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360585487Z namespaces/openshift-network-node-identity/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360693649Z namespaces/openshift-network-node-identity/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36074049Z namespaces/openshift-network-node-identity/core/serviceaccounts/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360791291Z namespaces/openshift-network-node-identity/core/serviceaccounts/network-node-identity.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360839133Z namespaces/openshift-network-node-identity/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360888614Z namespaces/openshift-network-node-identity/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360933665Z namespaces/openshift-network-node-identity/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.360980846Z namespaces/openshift-network-node-identity/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361031867Z namespaces/openshift-network-node-identity/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361081358Z namespaces/openshift-network-node-identity/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361179921Z namespaces/openshift-network-node-identity/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361265852Z namespaces/openshift-network-node-identity/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361315324Z namespaces/openshift-network-node-identity/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361354075Z namespaces/openshift-network-node-identity/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361410236Z namespaces/openshift-network-node-identity/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361440267Z namespaces/openshift-network-node-identity/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361489798Z namespaces/openshift-network-node-identity/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361497668Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361529409Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361686273Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.361867787Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3619998Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.362236995Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.362470181Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.362677276Z namespaces/openshift-network-node-identity/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.362717587Z namespaces/openshift-network-node-identity/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.362780839Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.362792609Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.362800509Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/rolebindings/network-node-identity-leases.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36285847Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.362910911Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/roles/network-node-identity-leases.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.362967193Z namespaces/openshift-network-node-identity/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.362988123Z namespaces/openshift-network-node-identity/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363055865Z namespaces/openshift-network-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363071065Z namespaces/openshift-network-operator/openshift-network-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363177358Z namespaces/openshift-network-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363201689Z namespaces/openshift-network-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36326188Z namespaces/openshift-network-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36328305Z namespaces/openshift-network-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363409623Z namespaces/openshift-network-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363489935Z namespaces/openshift-network-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363569337Z namespaces/openshift-network-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363619468Z namespaces/openshift-network-operator/apps/daemonsets/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363669789Z namespaces/openshift-network-operator/apps/daemonsets/iptables-alerter.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363727151Z namespaces/openshift-network-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363751282Z namespaces/openshift-network-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363811063Z namespaces/openshift-network-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363854174Z namespaces/openshift-network-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363934286Z namespaces/openshift-network-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.363974847Z namespaces/openshift-network-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.364027378Z namespaces/openshift-network-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36410449Z namespaces/openshift-network-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.364171441Z namespaces/openshift-network-operator/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.364184112Z namespaces/openshift-network-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.364230743Z namespaces/openshift-network-operator/coordination.k8s.io/leases/network-operator-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.364269904Z namespaces/openshift-network-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.364313335Z namespaces/openshift-network-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.364465028Z namespaces/openshift-network-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36454703Z namespaces/openshift-network-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.364705784Z namespaces/openshift-network-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.364787006Z namespaces/openshift-network-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36497406Z namespaces/openshift-network-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365070203Z namespaces/openshift-network-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365222496Z namespaces/openshift-network-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365277548Z namespaces/openshift-network-operator/core/configmaps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365316419Z namespaces/openshift-network-operator/core/configmaps/applied-cluster.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365396361Z namespaces/openshift-network-operator/core/configmaps/iptables-alerter-script.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365459212Z namespaces/openshift-network-operator/core/serviceaccounts/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365498093Z namespaces/openshift-network-operator/core/serviceaccounts/iptables-alerter.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365547404Z namespaces/openshift-network-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365586865Z namespaces/openshift-network-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365645516Z namespaces/openshift-network-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365678987Z namespaces/openshift-network-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365741049Z namespaces/openshift-network-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36578869Z namespaces/openshift-network-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365864772Z namespaces/openshift-network-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365953954Z namespaces/openshift-network-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.365995895Z namespaces/openshift-network-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366037876Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366091747Z namespaces/openshift-network-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366155099Z namespaces/openshift-network-operator/monitoring.coreos.com/prometheusrules/openshift-network-operator-ipsec-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366273641Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366312962Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors/network-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366369204Z namespaces/openshift-network-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366408865Z namespaces/openshift-network-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366456806Z namespaces/openshift-network-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366469056Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366500307Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36664884Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366825004Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.366965428Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367184563Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367412958Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367631694Z namespaces/openshift-network-operator/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367641024Z namespaces/openshift-network-operator/pods/iptables-alerter-4ln8q/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367669814Z namespaces/openshift-network-operator/pods/iptables-alerter-4ln8q/iptables-alerter-4ln8q.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367739106Z namespaces/openshift-network-operator/pods/iptables-alerter-4ln8q/iptables-alerter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367748806Z namespaces/openshift-network-operator/pods/iptables-alerter-4ln8q/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367753436Z namespaces/openshift-network-operator/pods/iptables-alerter-4ln8q/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367785287Z namespaces/openshift-network-operator/pods/iptables-alerter-4ln8q/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367880739Z namespaces/openshift-network-operator/pods/iptables-alerter-4ln8q/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367943361Z namespaces/openshift-network-operator/pods/iptables-alerter-4ln8q/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.367979502Z namespaces/openshift-network-operator/pods/iptables-alerter-j9mkq/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368026653Z namespaces/openshift-network-operator/pods/iptables-alerter-j9mkq/iptables-alerter-j9mkq.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368097114Z namespaces/openshift-network-operator/pods/iptables-alerter-j9mkq/iptables-alerter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368106135Z namespaces/openshift-network-operator/pods/iptables-alerter-j9mkq/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368110535Z namespaces/openshift-network-operator/pods/iptables-alerter-j9mkq/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368152386Z namespaces/openshift-network-operator/pods/iptables-alerter-j9mkq/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368256308Z namespaces/openshift-network-operator/pods/iptables-alerter-j9mkq/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36833386Z namespaces/openshift-network-operator/pods/iptables-alerter-j9mkq/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368353151Z namespaces/openshift-network-operator/pods/iptables-alerter-q6gt8/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368399912Z namespaces/openshift-network-operator/pods/iptables-alerter-q6gt8/iptables-alerter-q6gt8.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368471583Z namespaces/openshift-network-operator/pods/iptables-alerter-q6gt8/iptables-alerter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368482314Z namespaces/openshift-network-operator/pods/iptables-alerter-q6gt8/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368486584Z namespaces/openshift-network-operator/pods/iptables-alerter-q6gt8/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368527365Z namespaces/openshift-network-operator/pods/iptables-alerter-q6gt8/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368617057Z namespaces/openshift-network-operator/pods/iptables-alerter-q6gt8/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368691479Z namespaces/openshift-network-operator/pods/iptables-alerter-q6gt8/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368721629Z namespaces/openshift-network-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.36877267Z namespaces/openshift-network-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368814881Z namespaces/openshift-network-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368855012Z namespaces/openshift-network-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368908484Z namespaces/openshift-node/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368915064Z namespaces/openshift-node/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368929214Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.368965725Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.369146979Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.369325194Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.369464587Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.369675872Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.369911687Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370105192Z namespaces/openshift-operator-lifecycle-manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370172614Z namespaces/openshift-operator-lifecycle-manager/openshift-operator-lifecycle-manager.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370243075Z namespaces/openshift-operator-lifecycle-manager/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370280966Z namespaces/openshift-operator-lifecycle-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370327487Z namespaces/openshift-operator-lifecycle-manager/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370372358Z namespaces/openshift-operator-lifecycle-manager/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.37046377Z namespaces/openshift-operator-lifecycle-manager/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370540622Z namespaces/openshift-operator-lifecycle-manager/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370616074Z namespaces/openshift-operator-lifecycle-manager/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370665955Z namespaces/openshift-operator-lifecycle-manager/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370709806Z namespaces/openshift-operator-lifecycle-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370744837Z namespaces/openshift-operator-lifecycle-manager/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370802809Z namespaces/openshift-operator-lifecycle-manager/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.37087429Z namespaces/openshift-operator-lifecycle-manager/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370922531Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.370956132Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.371030194Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.371090765Z namespaces/openshift-operator-lifecycle-manager/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.371136146Z namespaces/openshift-operator-lifecycle-manager/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.371249339Z namespaces/openshift-operator-lifecycle-manager/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.371333041Z namespaces/openshift-operator-lifecycle-manager/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.371430853Z namespaces/openshift-operator-lifecycle-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.371511035Z namespaces/openshift-operator-lifecycle-manager/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.371586877Z namespaces/openshift-operator-lifecycle-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3717252Z namespaces/openshift-operator-lifecycle-manager/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.371912105Z namespaces/openshift-operator-lifecycle-manager/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.371981317Z namespaces/openshift-operator-lifecycle-manager/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372030518Z namespaces/openshift-operator-lifecycle-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372090329Z namespaces/openshift-operator-lifecycle-manager/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.37215325Z namespaces/openshift-operator-lifecycle-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372193322Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372243733Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372327415Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372405216Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372448177Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372499859Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.37254754Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372594881Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/prometheusrules/olm-alert-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372650942Z namespaces/openshift-operator-lifecycle-manager/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372695973Z namespaces/openshift-operator-lifecycle-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372765675Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372772225Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.372821696Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.37298027Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.373238096Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.37338671Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.373602605Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.373854191Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374057636Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374083606Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/operatorgroups/olm-operators.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374150178Z namespaces/openshift-operator-lifecycle-manager/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374216999Z namespaces/openshift-operator-lifecycle-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374268421Z namespaces/openshift-operator-lifecycle-manager/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374307261Z namespaces/openshift-operator-lifecycle-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374341392Z namespaces/openshift-operators/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374349722Z namespaces/openshift-operators/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374353933Z namespaces/openshift-operators/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374396614Z namespaces/openshift-operators/coordination.k8s.io/leases/sail-operator-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374452485Z namespaces/openshift-operators/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374464725Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374517277Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.37467973Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.374869535Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.375011918Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.375232954Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.375478029Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.375776506Z namespaces/openshift-operators/operators.coreos.com/installplans/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.375806067Z namespaces/openshift-operators/operators.coreos.com/installplans/install-blkrm.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.376208177Z namespaces/openshift-operators/operators.coreos.com/installplans/install-xwld4.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.376531184Z namespaces/openshift-operators/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.376574545Z namespaces/openshift-operators/operators.coreos.com/operatorconditions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.376630107Z namespaces/openshift-operators/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.376672138Z namespaces/openshift-operators/operators.coreos.com/operatorgroups/global-operators.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.376721199Z namespaces/openshift-operators/operators.coreos.com/subscriptions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.37676288Z namespaces/openshift-operators/operators.coreos.com/subscriptions/servicemeshoperator3.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.376831321Z namespaces/openshift-ovn-kubernetes/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.376864842Z namespaces/openshift-ovn-kubernetes/openshift-ovn-kubernetes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.376946914Z namespaces/openshift-ovn-kubernetes/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.376967485Z namespaces/openshift-ovn-kubernetes/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377021816Z namespaces/openshift-ovn-kubernetes/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377070117Z namespaces/openshift-ovn-kubernetes/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377277892Z namespaces/openshift-ovn-kubernetes/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377356514Z namespaces/openshift-ovn-kubernetes/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377427015Z namespaces/openshift-ovn-kubernetes/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377478947Z namespaces/openshift-ovn-kubernetes/apps/daemonsets/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377513957Z namespaces/openshift-ovn-kubernetes/apps/daemonsets/ovnkube-node.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377671361Z namespaces/openshift-ovn-kubernetes/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377714032Z namespaces/openshift-ovn-kubernetes/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377767023Z namespaces/openshift-ovn-kubernetes/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377799904Z namespaces/openshift-ovn-kubernetes/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377886146Z namespaces/openshift-ovn-kubernetes/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377927277Z namespaces/openshift-ovn-kubernetes/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.377974259Z namespaces/openshift-ovn-kubernetes/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.37805101Z namespaces/openshift-ovn-kubernetes/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.378115242Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.378144052Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.378180543Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/leases/ovn-kubernetes-master.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.378237325Z namespaces/openshift-ovn-kubernetes/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.378280586Z namespaces/openshift-ovn-kubernetes/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.378498751Z namespaces/openshift-ovn-kubernetes/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.378582393Z namespaces/openshift-ovn-kubernetes/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.37888757Z namespaces/openshift-ovn-kubernetes/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.379010183Z namespaces/openshift-ovn-kubernetes/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.379512595Z namespaces/openshift-ovn-kubernetes/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.379594567Z namespaces/openshift-ovn-kubernetes/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.379761321Z namespaces/openshift-ovn-kubernetes/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.379798862Z namespaces/openshift-ovn-kubernetes/core/configmaps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.379842933Z namespaces/openshift-ovn-kubernetes/core/configmaps/ovnkube-config.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.379939705Z namespaces/openshift-ovn-kubernetes/core/configmaps/ovnkube-script-lib.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380047257Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380078248Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ovn-kubernetes-control-plane.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380204371Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ovn-kubernetes-node.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380271183Z namespaces/openshift-ovn-kubernetes/core/services/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380313004Z namespaces/openshift-ovn-kubernetes/core/services/ovn-kubernetes-node.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380373295Z namespaces/openshift-ovn-kubernetes/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380416376Z namespaces/openshift-ovn-kubernetes/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380450187Z namespaces/openshift-ovn-kubernetes/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380493468Z namespaces/openshift-ovn-kubernetes/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380532969Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380540339Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/network-attachment-definitions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.38058498Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/network-attachment-definitions/default.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380642102Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380679283Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380763374Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380837726Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380884807Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.380934969Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.38099001Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381025501Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/prometheusrules/networking-rules.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381109993Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381236906Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors/monitor-ovn-node.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381281937Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381290357Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381335118Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/ovn.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.38141539Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/signer.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381463561Z namespaces/openshift-ovn-kubernetes/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381508652Z namespaces/openshift-ovn-kubernetes/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381563224Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381571704Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381609465Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381781109Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.381974333Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.382111257Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.382334122Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.382570117Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.382775052Z namespaces/openshift-ovn-kubernetes/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.382784552Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.382834974Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovnkube-node-6rbsp.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383008518Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-node/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383017438Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383021978Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383050979Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383184252Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383255464Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383271334Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383280934Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383294365Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383337335Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383433338Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.38350815Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383526Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/nbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.38354231Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/nbdb/nbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383614602Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/nbdb/nbdb/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383659713Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/nbdb/nbdb/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383761685Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383825447Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383859168Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/northd/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383867708Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/northd/northd/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383871848Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/northd/northd/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.383902309Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/northd/northd/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384005501Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384078483Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/northd/northd/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384106044Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-acl-logging/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384113684Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384140154Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384248467Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384493463Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384542894Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384553074Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-controller/ovn-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384558815Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384577725Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384835801Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384906253Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384952124Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovnkube-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384961604Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384967224Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.384996165Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.390616928Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.39069378Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.390724941Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/sbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.390735721Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/sbdb/sbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.390739641Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/sbdb/sbdb/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.390778162Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/sbdb/sbdb/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.390878964Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.390948846Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-6rbsp/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.390989107Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391053138Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovnkube-node-nxnnq.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391250033Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-node/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391261433Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391265043Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391307234Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391417387Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391480738Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391522259Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.39153072Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.39153513Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3915655Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391677063Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391745315Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391777775Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/nbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391783935Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/nbdb/nbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391787376Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/nbdb/nbdb/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391830637Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/nbdb/nbdb/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.391931229Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392001831Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392061482Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/northd/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392070072Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/northd/northd/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392077213Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/northd/northd/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392154094Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/northd/northd/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392252087Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392324288Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/northd/northd/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392356199Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-acl-logging/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392364129Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392367839Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.39240502Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392655476Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-acl-logging/ovn-acl-logging/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392726398Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392763819Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392771079Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-controller/ovn-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.392774819Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.3928069Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.393025295Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.393095227Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.393154508Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovnkube-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.393166268Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.393169908Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.39321656Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.397795688Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.39786554Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.397904Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/sbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.397916011Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/sbdb/sbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.397920581Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/sbdb/sbdb/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.397942732Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/sbdb/sbdb/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398047944Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398140936Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-nxnnq/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398163357Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398241068Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovnkube-node-qnlsv.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398402492Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-node/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398411163Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398415583Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398439893Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398573046Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398638238Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398670239Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398678779Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398682279Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.39871901Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398851413Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398931075Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398960905Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/nbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398968176Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/nbdb/nbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.398971776Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/nbdb/nbdb/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399004746Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/nbdb/nbdb/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399101109Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399202061Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399226982Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/northd/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399231192Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/northd/northd/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399234602Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/northd/northd/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399280433Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/northd/northd/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399373595Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399453227Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/northd/northd/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399492928Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-acl-logging/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399500608Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399504238Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399536129Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399800576Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399855487Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399862657Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-controller/ovn-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399866287Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.399904158Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.400164244Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.400232956Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.400275737Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovnkube-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.400282317Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.400285807Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.400323818Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.404994688Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.40506586Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405086601Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/sbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405094051Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/sbdb/sbdb/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405097431Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/sbdb/sbdb/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405177383Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/sbdb/sbdb/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405281005Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405345377Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-qnlsv/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405389048Z namespaces/openshift-ovn-kubernetes/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405424769Z namespaces/openshift-ovn-kubernetes/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.40547948Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.40548733Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405528601Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405616043Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/openshift-ovn-kubernetes-nodes-identity-limited.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405771517Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405823918Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405869609Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.405957342Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/openshift-ovn-kubernetes-node-limited.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406038983Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406088725Z namespaces/openshift-ovn-kubernetes/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406153436Z namespaces/openshift-ovn-kubernetes/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406210568Z namespaces/openshift-route-controller-manager/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406218028Z namespaces/openshift-route-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406221808Z namespaces/openshift-route-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406267529Z namespaces/openshift-route-controller-manager/coordination.k8s.io/leases/openshift-route-controllers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.40632072Z namespaces/openshift-route-controller-manager/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.40632724Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406371841Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406576766Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406786651Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.406930925Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.407189481Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.407458387Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.407703003Z namespaces/openshift-service-ca-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.407765214Z namespaces/openshift-service-ca-operator/openshift-service-ca-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.407831666Z namespaces/openshift-service-ca-operator/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.407881487Z namespaces/openshift-service-ca-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.407927718Z namespaces/openshift-service-ca-operator/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.407967799Z namespaces/openshift-service-ca-operator/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408050111Z namespaces/openshift-service-ca-operator/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408177794Z namespaces/openshift-service-ca-operator/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408285017Z namespaces/openshift-service-ca-operator/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408332858Z namespaces/openshift-service-ca-operator/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408369099Z namespaces/openshift-service-ca-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.40842087Z namespaces/openshift-service-ca-operator/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408464281Z namespaces/openshift-service-ca-operator/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408541083Z namespaces/openshift-service-ca-operator/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408588144Z namespaces/openshift-service-ca-operator/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408632275Z namespaces/openshift-service-ca-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408709916Z namespaces/openshift-service-ca-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408757978Z namespaces/openshift-service-ca-operator/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408764408Z namespaces/openshift-service-ca-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408805169Z namespaces/openshift-service-ca-operator/coordination.k8s.io/leases/service-ca-operator-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.40885604Z namespaces/openshift-service-ca-operator/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.408900841Z namespaces/openshift-service-ca-operator/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.409048855Z namespaces/openshift-service-ca-operator/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.409150927Z namespaces/openshift-service-ca-operator/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.409348712Z namespaces/openshift-service-ca-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.409431974Z namespaces/openshift-service-ca-operator/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.409551927Z namespaces/openshift-service-ca-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.40967823Z namespaces/openshift-service-ca-operator/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.409851923Z namespaces/openshift-service-ca-operator/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.409908825Z namespaces/openshift-service-ca-operator/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.409957406Z namespaces/openshift-service-ca-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410009287Z namespaces/openshift-service-ca-operator/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410054698Z namespaces/openshift-service-ca-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41009878Z namespaces/openshift-service-ca-operator/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410162461Z namespaces/openshift-service-ca-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410253843Z namespaces/openshift-service-ca-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410333095Z namespaces/openshift-service-ca-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410392707Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410443948Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410498819Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41054401Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors/service-ca-operator.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410594621Z namespaces/openshift-service-ca-operator/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410640492Z namespaces/openshift-service-ca-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410684763Z namespaces/openshift-service-ca-operator/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410691954Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410726384Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.410877608Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.411054542Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.411211036Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41140898Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.411652726Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.411849761Z namespaces/openshift-service-ca-operator/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.411856861Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-d6fc45fc5-k25j5/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.411893962Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-d6fc45fc5-k25j5/service-ca-operator-d6fc45fc5-k25j5.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.411972324Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-d6fc45fc5-k25j5/service-ca-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.411979894Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-d6fc45fc5-k25j5/service-ca-operator/service-ca-operator/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.411983534Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-d6fc45fc5-k25j5/service-ca-operator/service-ca-operator/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412019055Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-d6fc45fc5-k25j5/service-ca-operator/service-ca-operator/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412198069Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-d6fc45fc5-k25j5/service-ca-operator/service-ca-operator/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412360893Z namespaces/openshift-service-ca-operator/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412400424Z namespaces/openshift-service-ca-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412449175Z namespaces/openshift-service-ca-operator/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412492326Z namespaces/openshift-service-ca-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412538147Z namespaces/openshift-service-ca/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412580338Z namespaces/openshift-service-ca/openshift-service-ca.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41263958Z namespaces/openshift-service-ca/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412682221Z namespaces/openshift-service-ca/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412732602Z namespaces/openshift-service-ca/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412771943Z namespaces/openshift-service-ca/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412857425Z namespaces/openshift-service-ca/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.412969997Z namespaces/openshift-service-ca/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41306805Z namespaces/openshift-service-ca/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413113671Z namespaces/openshift-service-ca/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413220743Z namespaces/openshift-service-ca/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413270025Z namespaces/openshift-service-ca/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413317546Z namespaces/openshift-service-ca/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413398918Z namespaces/openshift-service-ca/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413443199Z namespaces/openshift-service-ca/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41348781Z namespaces/openshift-service-ca/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413568582Z namespaces/openshift-service-ca/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413617513Z namespaces/openshift-service-ca/coordination.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413624073Z namespaces/openshift-service-ca/coordination.k8s.io/leases/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413670324Z namespaces/openshift-service-ca/coordination.k8s.io/leases/service-ca-controller-lock.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413719595Z namespaces/openshift-service-ca/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413765856Z namespaces/openshift-service-ca/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413869089Z namespaces/openshift-service-ca/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.413950701Z namespaces/openshift-service-ca/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414060363Z namespaces/openshift-service-ca/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414161326Z namespaces/openshift-service-ca/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414277479Z namespaces/openshift-service-ca/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414409402Z namespaces/openshift-service-ca/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414584616Z namespaces/openshift-service-ca/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414636577Z namespaces/openshift-service-ca/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414683938Z namespaces/openshift-service-ca/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41473857Z namespaces/openshift-service-ca/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414785431Z namespaces/openshift-service-ca/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414827472Z namespaces/openshift-service-ca/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414877073Z namespaces/openshift-service-ca/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.414952495Z namespaces/openshift-service-ca/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.415033577Z namespaces/openshift-service-ca/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.415077978Z namespaces/openshift-service-ca/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.415134489Z namespaces/openshift-service-ca/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41518573Z namespaces/openshift-service-ca/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.415231651Z namespaces/openshift-service-ca/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.415278362Z namespaces/openshift-service-ca/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.415286703Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.415324883Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.415475857Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.415678612Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.415833715Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.416060991Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.416363838Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.416563903Z namespaces/openshift-service-ca/pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.416572423Z namespaces/openshift-service-ca/pods/service-ca-865cb79987-d5swp/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.416621904Z namespaces/openshift-service-ca/pods/service-ca-865cb79987-d5swp/service-ca-865cb79987-d5swp.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.416734077Z namespaces/openshift-service-ca/pods/service-ca-865cb79987-d5swp/service-ca-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.416743127Z namespaces/openshift-service-ca/pods/service-ca-865cb79987-d5swp/service-ca-controller/service-ca-controller/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.416747387Z namespaces/openshift-service-ca/pods/service-ca-865cb79987-d5swp/service-ca-controller/service-ca-controller/logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.416794268Z namespaces/openshift-service-ca/pods/service-ca-865cb79987-d5swp/service-ca-controller/service-ca-controller/logs/current.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417030524Z namespaces/openshift-service-ca/pods/service-ca-865cb79987-d5swp/service-ca-controller/service-ca-controller/logs/previous.insecure.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417116236Z namespaces/openshift-service-ca/pods/service-ca-865cb79987-d5swp/service-ca-controller/service-ca-controller/logs/previous.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417185397Z namespaces/openshift-service-ca/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417236869Z namespaces/openshift-service-ca/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41728656Z namespaces/openshift-service-ca/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417342381Z namespaces/openshift-service-ca/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417395702Z namespaces/openshift-user-workload-monitoring/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417450354Z namespaces/openshift-user-workload-monitoring/openshift-user-workload-monitoring.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417519825Z namespaces/openshift-user-workload-monitoring/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417567396Z namespaces/openshift-user-workload-monitoring/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417623838Z namespaces/openshift-user-workload-monitoring/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417677339Z namespaces/openshift-user-workload-monitoring/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417772141Z namespaces/openshift-user-workload-monitoring/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417868864Z namespaces/openshift-user-workload-monitoring/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.417967366Z namespaces/openshift-user-workload-monitoring/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418020697Z namespaces/openshift-user-workload-monitoring/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418078789Z namespaces/openshift-user-workload-monitoring/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41815194Z namespaces/openshift-user-workload-monitoring/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418197811Z namespaces/openshift-user-workload-monitoring/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418297504Z namespaces/openshift-user-workload-monitoring/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418357025Z namespaces/openshift-user-workload-monitoring/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418405426Z namespaces/openshift-user-workload-monitoring/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418499888Z namespaces/openshift-user-workload-monitoring/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.41855833Z namespaces/openshift-user-workload-monitoring/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418618201Z namespaces/openshift-user-workload-monitoring/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418728354Z namespaces/openshift-user-workload-monitoring/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418829236Z namespaces/openshift-user-workload-monitoring/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.418921858Z namespaces/openshift-user-workload-monitoring/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419021231Z namespaces/openshift-user-workload-monitoring/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419137374Z namespaces/openshift-user-workload-monitoring/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419287487Z namespaces/openshift-user-workload-monitoring/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419471422Z namespaces/openshift-user-workload-monitoring/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419537083Z namespaces/openshift-user-workload-monitoring/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419588524Z namespaces/openshift-user-workload-monitoring/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419649366Z namespaces/openshift-user-workload-monitoring/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419696087Z namespaces/openshift-user-workload-monitoring/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419755458Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419804809Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419899092Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.419996164Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.420058605Z namespaces/openshift-user-workload-monitoring/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.420114877Z namespaces/openshift-user-workload-monitoring/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.420191139Z namespaces/openshift-user-workload-monitoring/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.42023841Z namespaces/openshift-user-workload-monitoring/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.420299481Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.420306281Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.420345142Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.420509326Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.42069874Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.420852594Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.42107316Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.421356686Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.421588592Z namespaces/openshift-user-workload-monitoring/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.421617772Z namespaces/openshift-user-workload-monitoring/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.421737645Z namespaces/openshift-user-workload-monitoring/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.421747465Z namespaces/openshift-user-workload-monitoring/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.421755336Z namespaces/openshift/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.421809557Z namespaces/openshift/openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.421868118Z namespaces/openshift/apps.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.4219251Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.421968311Z namespaces/openshift/apps/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422003491Z namespaces/openshift/apps/daemonsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422087073Z namespaces/openshift/apps/deployments.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422195906Z namespaces/openshift/apps/replicasets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422276608Z namespaces/openshift/apps/statefulsets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422321679Z namespaces/openshift/autoscaling/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.4223527Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422405561Z namespaces/openshift/batch/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422442962Z namespaces/openshift/batch/cronjobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422522484Z namespaces/openshift/batch/jobs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422564715Z namespaces/openshift/build.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422611736Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422687698Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422735619Z namespaces/openshift/core/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.4227662Z namespaces/openshift/core/configmaps.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422876902Z namespaces/openshift/core/endpoints.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.422954054Z namespaces/openshift/core/events.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.423035996Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.423117458Z namespaces/openshift/core/pods.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.42322554Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.423357494Z namespaces/openshift/core/secrets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.423527398Z namespaces/openshift/core/services.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.423577829Z namespaces/openshift/discovery.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.423603009Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.423659661Z namespaces/openshift/image.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.423706182Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.424954881Z namespaces/openshift/image.openshift.io/imagestreams/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.424993282Z namespaces/openshift/image.openshift.io/imagestreams/cli-artifacts.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.425086785Z namespaces/openshift/image.openshift.io/imagestreams/cli.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.425251839Z namespaces/openshift/image.openshift.io/imagestreams/dotnet-runtime.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.425370581Z namespaces/openshift/image.openshift.io/imagestreams/dotnet.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.425485084Z namespaces/openshift/image.openshift.io/imagestreams/driver-toolkit.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.425575016Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-eap-openshift-java11.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.42572846Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-eap-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.425904634Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-java-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.426046197Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-java11-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.42616321Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-karaf-openshift-jdk11.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.426261663Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-karaf-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.426383305Z namespaces/openshift/image.openshift.io/imagestreams/golang.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.426484698Z namespaces/openshift/image.openshift.io/imagestreams/httpd.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.42659559Z namespaces/openshift/image.openshift.io/imagestreams/installer-artifacts.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.426689113Z namespaces/openshift/image.openshift.io/imagestreams/installer.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.426813045Z namespaces/openshift/image.openshift.io/imagestreams/java-runtime.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.426920938Z namespaces/openshift/image.openshift.io/imagestreams/java.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.427038661Z namespaces/openshift/image.openshift.io/imagestreams/jboss-datagrid73-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.427175944Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp3-openjdk11-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.427282487Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp3-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.427386059Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp4-openjdk11-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.427484441Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp4-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.427586214Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk11-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.427683586Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.427810329Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk8-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.427944492Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk8-runtime-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.428071475Z namespaces/openshift/image.openshift.io/imagestreams/jboss-webserver57-openjdk11-tomcat9-openshift-ubi8.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.428230449Z namespaces/openshift/image.openshift.io/imagestreams/jboss-webserver57-openjdk8-tomcat9-openshift-ubi8.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.428345802Z namespaces/openshift/image.openshift.io/imagestreams/jenkins-agent-base.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.428447684Z namespaces/openshift/image.openshift.io/imagestreams/jenkins.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.428555927Z namespaces/openshift/image.openshift.io/imagestreams/mariadb.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.428671329Z namespaces/openshift/image.openshift.io/imagestreams/must-gather.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.428758462Z namespaces/openshift/image.openshift.io/imagestreams/mysql.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.428874524Z namespaces/openshift/image.openshift.io/imagestreams/network-tools.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.428985767Z namespaces/openshift/image.openshift.io/imagestreams/nginx.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.429137671Z namespaces/openshift/image.openshift.io/imagestreams/nodejs.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.429291864Z namespaces/openshift/image.openshift.io/imagestreams/oauth-proxy.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.429388946Z namespaces/openshift/image.openshift.io/imagestreams/openjdk-11-rhel7.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.429508549Z namespaces/openshift/image.openshift.io/imagestreams/perl.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.429659763Z namespaces/openshift/image.openshift.io/imagestreams/php.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.429730285Z namespaces/openshift/image.openshift.io/imagestreams/postgresql.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.429862958Z namespaces/openshift/image.openshift.io/imagestreams/postgresql13-for-sso75-openshift-rhel8.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.42996157Z namespaces/openshift/image.openshift.io/imagestreams/postgresql13-for-sso76-openshift-rhel8.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.430052532Z namespaces/openshift/image.openshift.io/imagestreams/python.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.430202726Z namespaces/openshift/image.openshift.io/imagestreams/redhat-openjdk18-openshift.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.430335849Z namespaces/openshift/image.openshift.io/imagestreams/redis.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.430447142Z namespaces/openshift/image.openshift.io/imagestreams/ruby.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.430563164Z namespaces/openshift/image.openshift.io/imagestreams/sso75-openshift-rhel8.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.430659176Z namespaces/openshift/image.openshift.io/imagestreams/sso76-openshift-rhel8.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.430753499Z namespaces/openshift/image.openshift.io/imagestreams/tests.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.430847751Z namespaces/openshift/image.openshift.io/imagestreams/tools.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.430966914Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-11-runtime.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.431106897Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-11.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.431266671Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-17-runtime.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.431380494Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-17.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.431497027Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-21-runtime.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.431612079Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-21.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.431702401Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-8-runtime.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.431851095Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-8.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.431954837Z namespaces/openshift/k8s.ovn.org/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432006239Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432098091Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432208623Z namespaces/openshift/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432252074Z namespaces/openshift/monitoring.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432304486Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432354257Z namespaces/openshift/networking.k8s.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432391808Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432443379Z namespaces/openshift/operators.coreos.com/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432453479Z namespaces/openshift/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.43249725Z namespaces/openshift/operators.coreos.com/clusterserviceversions/authorino-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432648014Z namespaces/openshift/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.19.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432827128Z namespaces/openshift/operators.coreos.com/clusterserviceversions/dns-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.432980391Z namespaces/openshift/operators.coreos.com/clusterserviceversions/limitador-operator.v1.3.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.433284099Z namespaces/openshift/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.3.2.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.433526064Z namespaces/openshift/operators.coreos.com/clusterserviceversions/servicemeshoperator3.v3.1.0.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.433724969Z namespaces/openshift/policy/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.43376501Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.433800881Z namespaces/openshift/route.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.433850202Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.433904153Z namespaces/openshift/template.openshift.io/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.433913624Z namespaces/openshift/template.openshift.io/templates/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.433950005Z namespaces/openshift/template.openshift.io/templates/cache-service.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.434064047Z namespaces/openshift/template.openshift.io/templates/cakephp-mysql-example.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.434245681Z namespaces/openshift/template.openshift.io/templates/cakephp-mysql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.434382655Z namespaces/openshift/template.openshift.io/templates/dancer-mysql-example.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.434510148Z namespaces/openshift/template.openshift.io/templates/dancer-mysql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.434698652Z namespaces/openshift/template.openshift.io/templates/datagrid-service.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.434824685Z namespaces/openshift/template.openshift.io/templates/django-psql-example.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.434960078Z namespaces/openshift/template.openshift.io/templates/django-psql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.435090772Z namespaces/openshift/template.openshift.io/templates/eap-xp3-basic-s2i.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.435254645Z namespaces/openshift/template.openshift.io/templates/eap-xp4-basic-s2i.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.435386179Z namespaces/openshift/template.openshift.io/templates/eap74-basic-s2i.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.435514972Z namespaces/openshift/template.openshift.io/templates/eap74-https-s2i.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.435671485Z namespaces/openshift/template.openshift.io/templates/eap74-sso-s2i.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.435834599Z namespaces/openshift/template.openshift.io/templates/httpd-example.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.435954872Z namespaces/openshift/template.openshift.io/templates/jenkins-ephemeral-monitored.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.436099196Z namespaces/openshift/template.openshift.io/templates/jenkins-ephemeral.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.436245049Z namespaces/openshift/template.openshift.io/templates/jenkins-persistent-monitored.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.436379172Z namespaces/openshift/template.openshift.io/templates/jenkins-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.436504285Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk11-tomcat9-ubi8-basic-s2i.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.436645488Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk11-tomcat9-ubi8-https-s2i.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.436762941Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk8-tomcat9-ubi8-basic-s2i.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.436887564Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk8-tomcat9-ubi8-https-s2i.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.437026237Z namespaces/openshift/template.openshift.io/templates/mariadb-ephemeral.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.437178771Z namespaces/openshift/template.openshift.io/templates/mariadb-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.437300324Z namespaces/openshift/template.openshift.io/templates/mysql-ephemeral.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.437405746Z namespaces/openshift/template.openshift.io/templates/mysql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.43753649Z namespaces/openshift/template.openshift.io/templates/nginx-example.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.437652782Z namespaces/openshift/template.openshift.io/templates/nodejs-postgresql-example.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.437786415Z namespaces/openshift/template.openshift.io/templates/nodejs-postgresql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.437922719Z namespaces/openshift/template.openshift.io/templates/openjdk-web-basic-s2i.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.438045922Z namespaces/openshift/template.openshift.io/templates/postgresql-ephemeral.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.438191645Z namespaces/openshift/template.openshift.io/templates/postgresql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.438311288Z namespaces/openshift/template.openshift.io/templates/rails-pgsql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.438453411Z namespaces/openshift/template.openshift.io/templates/rails-postgresql-example.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.438596775Z namespaces/openshift/template.openshift.io/templates/react-web-app-example.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.438727738Z namespaces/openshift/template.openshift.io/templates/redis-ephemeral.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.43884579Z namespaces/openshift/template.openshift.io/templates/redis-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.438944663Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel-rest-3scale.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.439081886Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel-xml.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.439228259Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.439338152Z namespaces/openshift/template.openshift.io/templates/sso75-https.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.439458675Z namespaces/openshift/template.openshift.io/templates/sso75-ocp4-x509-https.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.439583168Z namespaces/openshift/template.openshift.io/templates/sso75-ocp4-x509-postgresql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.439717881Z namespaces/openshift/template.openshift.io/templates/sso75-postgresql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.439857654Z namespaces/openshift/template.openshift.io/templates/sso75-postgresql.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.439983057Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-https.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.440182712Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-postgresql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.440330966Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-postgresql.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.440466049Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-x509-https.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.440585172Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-x509-postgresql-persistent.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.440717595Z network_logs/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.440760336Z network_logs/cluster_scale [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.440844638Z network_logs/ippools.whereabouts.cni.cncf.io [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.44093954Z network_logs/multi-networkpolicy [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.441015422Z network_logs/net-attach-def [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.441097924Z network_logs/overlappingrangeipreservations.whereabouts.cni.cncf.io [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.441282378Z network_logs/ovn_kubernetes_top_pods [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.44137917Z network_logs/ovnk_database_store.tar.gz [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.443104711Z nodes/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.443164103Z nodes/debug [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.443267805Z nodes/ip-10-0-128-142.ec2.internal/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.443316936Z nodes/ip-10-0-128-142.ec2.internal/cpu_affinities.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.443591883Z nodes/ip-10-0-128-142.ec2.internal/dmesg [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.44387656Z nodes/ip-10-0-128-142.ec2.internal/ethtool_channels [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.443949171Z nodes/ip-10-0-128-142.ec2.internal/ethtool_features [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.444040724Z nodes/ip-10-0-128-142.ec2.internal/ip-10-0-128-142.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.444476114Z nodes/ip-10-0-128-142.ec2.internal/irq_affinities.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.444630328Z nodes/ip-10-0-128-142.ec2.internal/lscpu [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.444761281Z nodes/ip-10-0-128-142.ec2.internal/lspci [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.444856493Z nodes/ip-10-0-128-142.ec2.internal/podresources.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.444978136Z nodes/ip-10-0-128-142.ec2.internal/pods_info.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.445089709Z nodes/ip-10-0-128-142.ec2.internal/proc_cmdline [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.445196731Z nodes/ip-10-0-128-142.ec2.internal/sysinfo.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.446894371Z nodes/ip-10-0-128-142.ec2.internal/sysinfo.tgz [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.447055055Z nodes/ip-10-0-131-162.ec2.internal/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.447103896Z nodes/ip-10-0-131-162.ec2.internal/cpu_affinities.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.447441104Z nodes/ip-10-0-131-162.ec2.internal/dmesg [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.44769102Z nodes/ip-10-0-131-162.ec2.internal/ethtool_channels [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.447768912Z nodes/ip-10-0-131-162.ec2.internal/ethtool_features [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.447855674Z nodes/ip-10-0-131-162.ec2.internal/ip-10-0-131-162.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.448263224Z nodes/ip-10-0-131-162.ec2.internal/irq_affinities.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.448404227Z nodes/ip-10-0-131-162.ec2.internal/lscpu [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.44852158Z nodes/ip-10-0-131-162.ec2.internal/lspci [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.448618742Z nodes/ip-10-0-131-162.ec2.internal/podresources.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.448721075Z nodes/ip-10-0-131-162.ec2.internal/pods_info.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.448843087Z nodes/ip-10-0-131-162.ec2.internal/proc_cmdline [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.448933649Z nodes/ip-10-0-131-162.ec2.internal/sysinfo.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.450695361Z nodes/ip-10-0-131-162.ec2.internal/sysinfo.tgz [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.450864315Z nodes/ip-10-0-139-15.ec2.internal/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.450907366Z nodes/ip-10-0-139-15.ec2.internal/cpu_affinities.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.451229294Z nodes/ip-10-0-139-15.ec2.internal/dmesg [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.45147976Z nodes/ip-10-0-139-15.ec2.internal/ethtool_channels [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.451558352Z nodes/ip-10-0-139-15.ec2.internal/ethtool_features [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.451641804Z nodes/ip-10-0-139-15.ec2.internal/ip-10-0-139-15.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.452013122Z nodes/ip-10-0-139-15.ec2.internal/irq_affinities.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.452177086Z nodes/ip-10-0-139-15.ec2.internal/lscpu [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.452299759Z nodes/ip-10-0-139-15.ec2.internal/lspci [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.452392502Z nodes/ip-10-0-139-15.ec2.internal/podresources.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.452497704Z nodes/ip-10-0-139-15.ec2.internal/pods_info.json [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.452618097Z nodes/ip-10-0-139-15.ec2.internal/proc_cmdline [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.452705479Z nodes/ip-10-0-139-15.ec2.internal/sysinfo.log [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.454488451Z nodes/ip-10-0-139-15.ec2.internal/sysinfo.tgz [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.454641755Z pod_network_connectivity_check/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.454686166Z pod_network_connectivity_check/podnetworkconnectivitychecks.yaml [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.454787948Z static-pods/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.454797559Z static-pods/kube-apiserver/ [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.46162262Z [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.461636861Z sent 70,560 bytes received 6,691,280 bytes 4,507,893.33 bytes/sec [must-gather] [must-gather-5jw29] OUT 2026-04-24T22:01:40.461641361Z total size is 81,192,498 speedup is 12.01 [must-gather] [must-gather ] OUT 2026-04-24T22:01:40.756988507Z namespace/openshift-must-gather-xhf46 deleted [must-gather] [must-gather] [must-gather] Reprinting Cluster State: [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 16dc56fd-55e9-4d08-86d8-6623a3d5ebfb [must-gather] ClientVersion: 4.21.5 [must-gather] ClusterVersion: Stable at "4.20.19" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [git-push-artifacts] WORK_DIR: /workspace/odh-ci-artifacts [git-push-artifacts] REPO_PATH: opendatahub-io/odh-build-metadata [git-push-artifacts] REPO_BRANCH: ci-artifacts [git-push-artifacts] SPARSE_FILE_PATH: test-artifacts/docs [git-push-artifacts] SOURCE_PATH: /workspace/artifacts-dir [git-push-artifacts] DEST_PATH: test-artifacts/kserve-group-test-vrgmp [git-push-artifacts] ALWAYS_PASS: false [git-push-artifacts] configuring gh token [git-push-artifacts] taking github token from Konflux bot [git-push-artifacts] Initialized empty Git repository in /workspace/odh-ci-artifacts/.git/ [git-push-artifacts] Using partial fetch with sparse checkout for: test-artifacts/docs [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] * [new branch] ci-artifacts -> origin/ci-artifacts [git-push-artifacts] Already on 'ci-artifacts' [git-push-artifacts] branch 'ci-artifacts' set up to track 'origin/ci-artifacts'. [git-push-artifacts] TASK_NAME=kserve-group-test-vrgmp-e2e-llm-inference-service [git-push-artifacts] PIPELINERUN_NAME=kserve-group-test-vrgmp [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] Already up to date. [git-push-artifacts] -rw-r--r--. 1 root 1001540000 14964986 Apr 24 22:03 /workspace/odh-ci-artifacts/test-artifacts/kserve-group-test-vrgmp/e2e-llm-inference-service.tar.gz [git-push-artifacts] [ci-artifacts 8efbc61] Updating CI Artifacts in e2e-llm-inference-service [git-push-artifacts] 1 file changed, 0 insertions(+), 0 deletions(-) [git-push-artifacts] create mode 100644 test-artifacts/kserve-group-test-vrgmp/e2e-llm-inference-service.tar.gz [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] Already up to date. [git-push-artifacts] To https://github.com/opendatahub-io/odh-build-metadata.git [git-push-artifacts] 95f2bac..8efbc61 ci-artifacts -> ci-artifacts [fail-if-needed] Failing pipeline because deploy-and-e2e step failed container step-fail-if-needed has failed : [{"key":"StartedAt","value":"2026-04-24T22:03:05.197Z","type":3}]