task e2e-llm-inference-service has failed: "step-fail-if-needed" exited with code 1: Error [get-kubeconfig] Found kubeconfig secret: cluster-xsp4k-admin-kubeconfig [get-kubeconfig] Wrote kubeconfig to /credentials/cluster-xsp4k-kubeconfig [get-kubeconfig] Found admin password secret: cluster-xsp4k-admin-password [get-kubeconfig] Retrieved username [get-kubeconfig] Wrote password to /credentials/cluster-xsp4k-password [get-kubeconfig] API Server URL: https://a59be8ef1618e4ac8b5f728e41b05ca4-17ea8ece44ba2f3a.elb.us-east-1.amazonaws.com:6443 [get-kubeconfig] Console URL: https://console-openshift-console.apps.c8cc9e53-dbe8-46dd-92c8-d1cbe7ed44ac.prod.konfluxeaas.com [clone-repo] feat/llmisvc/publisher-path-rbac [clone-repo] https://github.com/bartoszmajsak/kserve [clone-repo] Cloning into '/workspace/source'... [clone-repo] Updating files: 81% (2565/3137) Updating files: 82% (2573/3137) Updating files: 83% (2604/3137) Updating files: 84% (2636/3137) Updating files: 85% (2667/3137) Updating files: 86% (2698/3137) Updating files: 87% (2730/3137) Updating files: 88% (2761/3137) Updating files: 89% (2792/3137) Updating files: 90% (2824/3137) Updating files: 91% (2855/3137) Updating files: 92% (2887/3137) Updating files: 93% (2918/3137) Updating files: 94% (2949/3137) Updating files: 95% (2981/3137) Updating files: 96% (3012/3137) Updating files: 97% (3043/3137) Updating files: 98% (3075/3137) Updating files: 99% (3106/3137) Updating files: 100% (3137/3137) Updating files: 100% (3137/3137), done. [e2e-llm-inference-service] + bash [e2e-llm-inference-service] + STATUS_FILE=/test-status/deploy-and-e2e-status [e2e-llm-inference-service] + echo failed [e2e-llm-inference-service] + COMPONENT_NAME=kserve-agent-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-agent-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:42ffb1a3fbaec62e57c71970d909bef17aa5b27605c06189276dc315c300eb9f [e2e-llm-inference-service] + KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:42ffb1a3fbaec62e57c71970d909bef17aa5b27605c06189276dc315c300eb9f [e2e-llm-inference-service] + COMPONENT_NAME=kserve-controller-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-controller-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:7912d58f195183d9e721116618865eb65dab4d9d594a55ba497d4966a5572fda [e2e-llm-inference-service] + KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:7912d58f195183d9e721116618865eb65dab4d9d594a55ba497d4966a5572fda [e2e-llm-inference-service] + COMPONENT_NAME=kserve-router-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-router-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:4979cb407a123d624c47a64b3f1979504d753f7cd80622e43a8523addc69e145 [e2e-llm-inference-service] + KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:4979cb407a123d624c47a64b3f1979504d753f7cd80622e43a8523addc69e145 [e2e-llm-inference-service] + COMPONENT_NAME=kserve-storage-initializer-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-storage-initializer-ci '.[$component_name].image' [e2e-llm-inference-service] + export STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a [e2e-llm-inference-service] + STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a [e2e-llm-inference-service] + COMPONENT_NAME=odh-kserve-llmisvc-controller-ci [e2e-llm-inference-service] ++ jq -r --arg component_name odh-kserve-llmisvc-controller-ci '.[$component_name].image' [e2e-llm-inference-service] + export LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:d69df2f43218d8e9f989e3e2daf80e41a50953a20f5e2534497cb1de5b055298 [e2e-llm-inference-service] + LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:d69df2f43218d8e9f989e3e2daf80e41a50953a20f5e2534497cb1de5b055298 [e2e-llm-inference-service] + ./test/scripts/openshift-ci/run-e2e-tests.sh 'llminferenceservice and cluster_cpu and not autoscaling and not tracing' 2 llm-d [e2e-llm-inference-service] INFERENCE_POOL_GROUP=inference.networking.k8s.io (detected from OCP 4.21.24) [e2e-llm-inference-service] Installing on cluster [e2e-llm-inference-service] Using namespace: kserve for KServe components [e2e-llm-inference-service] SKLEARN_IMAGE=quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1744 [e2e-llm-inference-service] OPT_125M_MODEL_URI=s3://example-models/facebook/opt-125m [e2e-llm-inference-service] ERROR_404_ISVC_IMAGE=quay.io/opendatahub/error-404-isvc:odh-pr-1744 [e2e-llm-inference-service] SUCCESS_200_ISVC_IMAGE=quay.io/opendatahub/success-200-isvc:odh-pr-1744 [e2e-llm-inference-service] [INFO] Installing Kustomize v5.8.1 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed Kustomize v5.8.1 to /workspace/source/bin/kustomize [e2e-llm-inference-service] v5.8.1 [e2e-llm-inference-service] make: Entering directory '/workspace/source' [e2e-llm-inference-service] [INFO] Installing yq v4.52.1 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed yq v4.52.1 to /workspace/source/bin/yq [e2e-llm-inference-service] yq (https://github.com/mikefarah/yq/) version v4.52.1 [e2e-llm-inference-service] make: Leaving directory '/workspace/source' [e2e-llm-inference-service] Installing KServe Python SDK ... [e2e-llm-inference-service] [INFO] Installing uv 0.7.8 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed uv 0.7.8 to /workspace/source/bin/uv [e2e-llm-inference-service] warning: Failed to read project metadata (No `pyproject.toml` found in current directory or any parent directory). Running `uv self version` for compatibility. This fallback will be removed in the future; pass `--preview` to force an error. [e2e-llm-inference-service] uv 0.7.8 [e2e-llm-inference-service] Creating virtual environment... [e2e-llm-inference-service] warning: virtualenv's `--clear` has no effect (uv always clears the virtual environment) [e2e-llm-inference-service] Using CPython 3.9.25 interpreter at: /usr/bin/python3 [e2e-llm-inference-service] Creating virtual environment at: .venv [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] Using CPython 3.11.13 interpreter at: /usr/bin/python3.11 [e2e-llm-inference-service] Creating virtual environment at: .venv [e2e-llm-inference-service] Resolved 266 packages in 1ms [e2e-llm-inference-service] Building kserve @ file:///workspace/source/python/kserve [e2e-llm-inference-service] Downloading grpcio-tools (2.5MiB) [e2e-llm-inference-service] Downloading pydantic-core (2.0MiB) [e2e-llm-inference-service] Downloading grpcio (6.4MiB) [e2e-llm-inference-service] Downloading setuptools (1.2MiB) [e2e-llm-inference-service] Downloading cryptography (4.3MiB) [e2e-llm-inference-service] Downloading black (1.6MiB) [e2e-llm-inference-service] Downloading pandas (12.5MiB) [e2e-llm-inference-service] Downloading kubernetes (1.9MiB) [e2e-llm-inference-service] Downloading uvloop (3.8MiB) [e2e-llm-inference-service] Downloading numpy (15.7MiB) [e2e-llm-inference-service] Downloading aiohttp (1.7MiB) [e2e-llm-inference-service] Downloading mypy (17.2MiB) [e2e-llm-inference-service] Downloading portforward (3.9MiB) [e2e-llm-inference-service] Downloading botocore (12.9MiB) [e2e-llm-inference-service] Downloading pyarrow (40.1MiB) [e2e-llm-inference-service] Building timeout-sampler==1.0.3 [e2e-llm-inference-service] Building python-simple-logger==2.0.19 [e2e-llm-inference-service] Downloading aiohttp [e2e-llm-inference-service] Downloading pydantic-core [e2e-llm-inference-service] Downloading black [e2e-llm-inference-service] Downloading grpcio-tools [e2e-llm-inference-service] Downloading setuptools [e2e-llm-inference-service] Built python-simple-logger==2.0.19 [e2e-llm-inference-service] Downloading portforward [e2e-llm-inference-service] Downloading uvloop [e2e-llm-inference-service] Downloading cryptography [e2e-llm-inference-service] Downloading grpcio [e2e-llm-inference-service] Downloading kubernetes [e2e-llm-inference-service] Built timeout-sampler==1.0.3 [e2e-llm-inference-service] Downloading numpy [e2e-llm-inference-service] Built kserve @ file:///workspace/source/python/kserve [e2e-llm-inference-service] Downloading pandas [e2e-llm-inference-service] Downloading botocore [e2e-llm-inference-service] Downloading pyarrow [e2e-llm-inference-service] Downloading mypy [e2e-llm-inference-service] Prepared 101 packages in 1.85s [e2e-llm-inference-service] warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance. [e2e-llm-inference-service] If the cache and target directories are on different filesystems, hardlinking may not be supported. [e2e-llm-inference-service] If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning. [e2e-llm-inference-service] Installed 101 packages in 293ms [e2e-llm-inference-service] + aiohappyeyeballs==2.6.1 [e2e-llm-inference-service] + aiohttp==3.13.3 [e2e-llm-inference-service] + aiosignal==1.4.0 [e2e-llm-inference-service] + annotated-doc==0.0.4 [e2e-llm-inference-service] + annotated-types==0.7.0 [e2e-llm-inference-service] + anyio==4.9.0 [e2e-llm-inference-service] + attrs==25.3.0 [e2e-llm-inference-service] + avro==1.12.0 [e2e-llm-inference-service] + black==24.3.0 [e2e-llm-inference-service] + boto3==1.37.35 [e2e-llm-inference-service] + botocore==1.37.35 [e2e-llm-inference-service] + cachetools==5.5.2 [e2e-llm-inference-service] + certifi==2025.1.31 [e2e-llm-inference-service] + cffi==2.0.0 [e2e-llm-inference-service] + charset-normalizer==3.4.1 [e2e-llm-inference-service] + click==8.1.8 [e2e-llm-inference-service] + cloudevents==1.11.0 [e2e-llm-inference-service] + colorama==0.4.6 [e2e-llm-inference-service] + colorlog==6.10.1 [e2e-llm-inference-service] + coverage==7.8.0 [e2e-llm-inference-service] + cryptography==46.0.5 [e2e-llm-inference-service] + deprecation==2.1.0 [e2e-llm-inference-service] + durationpy==0.9 [e2e-llm-inference-service] + execnet==2.1.1 [e2e-llm-inference-service] + fastapi==0.136.3 [e2e-llm-inference-service] + frozenlist==1.5.0 [e2e-llm-inference-service] + google-auth==2.39.0 [e2e-llm-inference-service] + grpc-interceptor==0.15.4 [e2e-llm-inference-service] + grpcio==1.78.1 [e2e-llm-inference-service] + grpcio-testing==1.78.1 [e2e-llm-inference-service] + grpcio-tools==1.78.1 [e2e-llm-inference-service] + h11==0.16.0 [e2e-llm-inference-service] + httpcore==1.0.9 [e2e-llm-inference-service] + httptools==0.6.4 [e2e-llm-inference-service] + httpx==0.27.2 [e2e-llm-inference-service] + httpx-retries==0.4.5 [e2e-llm-inference-service] + idna==3.10 [e2e-llm-inference-service] + iniconfig==2.1.0 [e2e-llm-inference-service] + jinja2==3.1.6 [e2e-llm-inference-service] + jmespath==1.0.1 [e2e-llm-inference-service] + kserve==0.19.0 (from file:///workspace/source/python/kserve) [e2e-llm-inference-service] + kubernetes==32.0.1 [e2e-llm-inference-service] + markupsafe==3.0.2 [e2e-llm-inference-service] + multidict==6.4.3 [e2e-llm-inference-service] + mypy==0.991 [e2e-llm-inference-service] + mypy-extensions==1.0.0 [e2e-llm-inference-service] + numpy==2.2.4 [e2e-llm-inference-service] + oauthlib==3.2.2 [e2e-llm-inference-service] + orjson==3.10.16 [e2e-llm-inference-service] + packaging==24.2 [e2e-llm-inference-service] + pandas==2.2.3 [e2e-llm-inference-service] + pathspec==0.12.1 [e2e-llm-inference-service] + platformdirs==4.3.7 [e2e-llm-inference-service] + pluggy==1.5.0 [e2e-llm-inference-service] + portforward==0.7.1 [e2e-llm-inference-service] + prometheus-client==0.21.1 [e2e-llm-inference-service] + propcache==0.3.1 [e2e-llm-inference-service] + protobuf==6.33.5 [e2e-llm-inference-service] + psutil==5.9.8 [e2e-llm-inference-service] + pyarrow==19.0.1 [e2e-llm-inference-service] + pyasn1==0.6.3 [e2e-llm-inference-service] + pyasn1-modules==0.4.2 [e2e-llm-inference-service] + pycparser==2.22 [e2e-llm-inference-service] + pydantic==2.12.4 [e2e-llm-inference-service] + pydantic-core==2.41.5 [e2e-llm-inference-service] + pyjwt==2.12.1 [e2e-llm-inference-service] + pytest==7.4.4 [e2e-llm-inference-service] + pytest-asyncio==0.23.8 [e2e-llm-inference-service] + pytest-cov==5.0.0 [e2e-llm-inference-service] + pytest-httpx==0.30.0 [e2e-llm-inference-service] + pytest-json-report==1.5.0 [e2e-llm-inference-service] + pytest-metadata==3.1.1 [e2e-llm-inference-service] + pytest-xdist==3.6.1 [e2e-llm-inference-service] + python-dateutil==2.9.0.post0 [e2e-llm-inference-service] + python-dotenv==1.1.0 [e2e-llm-inference-service] + python-multipart==0.0.22 [e2e-llm-inference-service] + python-simple-logger==2.0.19 [e2e-llm-inference-service] + pytz==2025.2 [e2e-llm-inference-service] + pyyaml==6.0.2 [e2e-llm-inference-service] + requests==2.32.3 [e2e-llm-inference-service] + requests-oauthlib==2.0.0 [e2e-llm-inference-service] + rsa==4.9.1 [e2e-llm-inference-service] + s3transfer==0.11.4 [e2e-llm-inference-service] + setuptools==78.1.0 [e2e-llm-inference-service] + six==1.17.0 [e2e-llm-inference-service] + sniffio==1.3.1 [e2e-llm-inference-service] + starlette==1.2.1 [e2e-llm-inference-service] + tabulate==0.9.0 [e2e-llm-inference-service] + timeout-sampler==1.0.3 [e2e-llm-inference-service] + timing-asgi==0.3.1 [e2e-llm-inference-service] + tomlkit==0.13.2 [e2e-llm-inference-service] + typing-extensions==4.15.0 [e2e-llm-inference-service] + typing-inspection==0.4.2 [e2e-llm-inference-service] + tzdata==2025.2 [e2e-llm-inference-service] + urllib3==2.6.2 [e2e-llm-inference-service] + uvicorn==0.34.1 [e2e-llm-inference-service] + uvloop==0.21.0 [e2e-llm-inference-service] + watchfiles==1.0.5 [e2e-llm-inference-service] + websocket-client==1.8.0 [e2e-llm-inference-service] + websockets==15.0.1 [e2e-llm-inference-service] + yarl==1.20.0 [e2e-llm-inference-service] Audited 1 package in 46ms [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] [INFO] Installing Kustomize v5.8.1 for linux/amd64... [e2e-llm-inference-service] [INFO] Kustomize v5.8.1 is already installed in /workspace/source/bin (>= v5.8.1) [e2e-llm-inference-service] make: Entering directory '/workspace/source' [e2e-llm-inference-service] make: Leaving directory '/workspace/source' [e2e-llm-inference-service] Now using project "kserve" on server "https://a59be8ef1618e4ac8b5f728e41b05ca4-17ea8ece44ba2f3a.elb.us-east-1.amazonaws.com:6443". [e2e-llm-inference-service] [e2e-llm-inference-service] You can add applications to this project with the 'new-app' command. For example, try: [e2e-llm-inference-service] [e2e-llm-inference-service] oc new-app rails-postgresql-example [e2e-llm-inference-service] [e2e-llm-inference-service] to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application: [e2e-llm-inference-service] [e2e-llm-inference-service] kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname [e2e-llm-inference-service] [e2e-llm-inference-service] [INFO] Installing Kustomize v5.8.1 for linux/amd64... [e2e-llm-inference-service] [INFO] Kustomize v5.8.1 is already installed in /workspace/source/bin (>= v5.8.1) [e2e-llm-inference-service] make: Entering directory '/workspace/source' [e2e-llm-inference-service] make: Leaving directory '/workspace/source' [e2e-llm-inference-service] Creating namespace openshift-keda... [e2e-llm-inference-service] namespace/openshift-keda created [e2e-llm-inference-service] Namespace openshift-keda created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Creating OperatorGroup openshift-keda... [e2e-llm-inference-service] operatorgroup.operators.coreos.com/openshift-keda created [e2e-llm-inference-service] OperatorGroup openshift-keda created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Creating Subscription for openshift-custom-metrics-autoscaler-operator... [e2e-llm-inference-service] subscription.operators.coreos.com/openshift-custom-metrics-autoscaler-operator created [e2e-llm-inference-service] Subscription openshift-custom-metrics-autoscaler-operator created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Waiting for openshift-custom-metrics-autoscaler-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (0/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (5/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (10/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (15/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (20/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 found, but not yet Succeeded (Phase: Installing). Waiting... (25/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 found, but not yet Succeeded (Phase: Installing). Waiting... (30/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 found, but not yet Succeeded (Phase: Installing). Waiting... (35/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 found, but not yet Succeeded (Phase: Installing). Waiting... (40/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 found, but not yet Succeeded (Phase: Installing). Waiting... (45/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 is ready (Phase: Succeeded). [e2e-llm-inference-service] --- [e2e-llm-inference-service] Applying KedaController custom resource... [e2e-llm-inference-service] Warning: resource kedacontrollers/keda is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically. [e2e-llm-inference-service] kedacontroller.keda.sh/keda configured [e2e-llm-inference-service] KedaController custom resource applied. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Allowing time for KEDA components to be provisioned by the operator ... [e2e-llm-inference-service] Waiting for KEDA Operator pod (selector: "app=keda-operator") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-operator" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-operator" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-operator" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-operator-77f648d694-zdb27 1/1 Running 0 43s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-operator" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-operator-77f648d694-zdb27 condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-operator" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Operator pod is ready. [e2e-llm-inference-service] Waiting for KEDA Metrics API Server pod (selector: "app=keda-metrics-apiserver") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-metrics-apiserver" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-metrics-apiserver" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-metrics-apiserver" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-metrics-apiserver-55557df8c-cr42k 1/1 Running 0 47s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-metrics-apiserver" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-metrics-apiserver-55557df8c-cr42k condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-metrics-apiserver" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Metrics API Server pod is ready. [e2e-llm-inference-service] Waiting for KEDA Webhook pod (selector: "app=keda-admission-webhooks") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-admission-webhooks" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-admission-webhooks" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-admission-webhooks" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-admission-56f9798bb4-5k44d 1/1 Running 0 52s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-admission-webhooks" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-admission-56f9798bb4-5k44d condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-admission-webhooks" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Webhook pod is ready. [e2e-llm-inference-service] --- [e2e-llm-inference-service] ✅ KEDA deployment script finished successfully. [e2e-llm-inference-service] KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:7912d58f195183d9e721116618865eb65dab4d9d594a55ba497d4966a5572fda [e2e-llm-inference-service] LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:d69df2f43218d8e9f989e3e2daf80e41a50953a20f5e2534497cb1de5b055298 [e2e-llm-inference-service] KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:42ffb1a3fbaec62e57c71970d909bef17aa5b27605c06189276dc315c300eb9f [e2e-llm-inference-service] KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:4979cb407a123d624c47a64b3f1979504d753f7cd80622e43a8523addc69e145 [e2e-llm-inference-service] STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a [e2e-llm-inference-service] Installing KServe via kustomize... [e2e-llm-inference-service] === Final params.env [e2e-llm-inference-service] kserve-controller=quay.io/opendatahub/kserve-controller@sha256:7912d58f195183d9e721116618865eb65dab4d9d594a55ba497d4966a5572fda [e2e-llm-inference-service] llmisvc-controller=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:d69df2f43218d8e9f989e3e2daf80e41a50953a20f5e2534497cb1de5b055298 [e2e-llm-inference-service] kserve-agent=quay.io/opendatahub/kserve-agent@sha256:42ffb1a3fbaec62e57c71970d909bef17aa5b27605c06189276dc315c300eb9f [e2e-llm-inference-service] kserve-router=quay.io/opendatahub/kserve-router@sha256:4979cb407a123d624c47a64b3f1979504d753f7cd80622e43a8523addc69e145 [e2e-llm-inference-service] kserve-storage-initializer=quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a [e2e-llm-inference-service] kserve-llm-d=registry.redhat.io/rhaiis/vllm-cuda-rhel9@sha256:fc68d623d1bfc36c8cb2fe4a71f19c8578cfb420ce8ce07b20a02c1ee0be0cf3 [e2e-llm-inference-service] kserve-llm-d-inference-scheduler=quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0 [e2e-llm-inference-service] kserve-llm-d-routing-sidecar=quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0 [e2e-llm-inference-service] kserve-llm-d-uds-tokenizer=quay.io/opendatahub/llm-d-kv-cache:v0.8.0 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda=registry.redhat.io/rhaiis/vllm-cuda-rhel9@sha256:fc68d623d1bfc36c8cb2fe4a71f19c8578cfb420ce8ce07b20a02c1ee0be0cf3 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda-fast-1=registry.redhat.io/rhaiis/vllm-cuda-rhel9@sha256:fc68d623d1bfc36c8cb2fe4a71f19c8578cfb420ce8ce07b20a02c1ee0be0cf3 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda-fast-2=registry.redhat.io/rhaiis/vllm-cuda-rhel9@sha256:fc68d623d1bfc36c8cb2fe4a71f19c8578cfb420ce8ce07b20a02c1ee0be0cf3 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda-fast-1-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda-fast-2-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-amd-rocm=registry.redhat.io/rhaiis/vllm-rocm-rhel9@sha256:d9a48add238cc095fa43eeee17c8c4d104de60c4dc623e0bc7f8c4b53b2b2e97 [e2e-llm-inference-service] kserve-llm-d-amd-rocm-fast-1=registry.redhat.io/rhaiis/vllm-rocm-rhel9@sha256:d9a48add238cc095fa43eeee17c8c4d104de60c4dc623e0bc7f8c4b53b2b2e97 [e2e-llm-inference-service] kserve-llm-d-amd-rocm-fast-2=registry.redhat.io/rhaiis/vllm-rocm-rhel9@sha256:d9a48add238cc095fa43eeee17c8c4d104de60c4dc623e0bc7f8c4b53b2b2e97 [e2e-llm-inference-service] kserve-llm-d-amd-rocm-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-amd-rocm-fast-1-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-amd-rocm-fast-2-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi=registry.redhat.io/rhaii-early-access/vllm-gaudi-rhel9:3.4.0-ea.2 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi-fast-1=registry.redhat.io/rhaii-early-access/vllm-gaudi-rhel9:3.4.0-ea.2 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi-fast-2=registry.redhat.io/rhaii-early-access/vllm-gaudi-rhel9:3.4.0-ea.2 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi-upstream-version=0.16.0 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi-fast-1-upstream-version=0.16.0 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi-fast-2-upstream-version=0.16.0 [e2e-llm-inference-service] kserve-llm-d-ibm-spyre=registry.redhat.io/rhaiis/vllm-spyre-rhel9@sha256:80ae3e435a5be2c1f117f36599103ab05357917dd6e37f0df6613cb3ac2c13ea [e2e-llm-inference-service] kserve-llm-d-ibm-spyre-fast-1=registry.redhat.io/rhaiis/vllm-spyre-rhel9@sha256:80ae3e435a5be2c1f117f36599103ab05357917dd6e37f0df6613cb3ac2c13ea [e2e-llm-inference-service] kserve-llm-d-ibm-spyre-fast-2=registry.redhat.io/rhaiis/vllm-spyre-rhel9@sha256:80ae3e435a5be2c1f117f36599103ab05357917dd6e37f0df6613cb3ac2c13ea [e2e-llm-inference-service] kserve-llm-d-ibm-spyre-upstream-version=0.10.2 [e2e-llm-inference-service] kserve-llm-d-ibm-spyre-fast-1-upstream-version=0.10.2 [e2e-llm-inference-service] kserve-llm-d-ibm-spyre-fast-2-upstream-version=0.10.2 [e2e-llm-inference-service] # TODO update when our changes are introduced in the official image [e2e-llm-inference-service] kube-rbac-proxy=quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3 [e2e-llm-inference-service] kserve-localmodel-controller=quay.io/opendatahub/odh-kserve-localmodel-controller:odh-master [e2e-llm-inference-service] kserve-localmodelnode-agent=quay.io/opendatahub/odh-kserve-localmodelnode-agent:odh-master# Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencemodelrewrites.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceobjectives.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepoolimports.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied [e2e-llm-inference-service] Waiting for CRDs to be established... [e2e-llm-inference-service] Waiting for CRD inferenceservices.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD inferenceservices.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD llminferenceserviceconfigs.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD llminferenceserviceconfigs.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD clusterstoragecontainers.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD clusterstoragecontainers.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD datascienceclusters.datasciencecluster.opendatahub.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD datascienceclusters.datasciencecluster.opendatahub.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io condition met [e2e-llm-inference-service] Applying all resources... [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencemodelrewrites.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceobjectives.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepoolimports.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied [e2e-llm-inference-service] serviceaccount/kserve-controller-manager serverside-applied [e2e-llm-inference-service] serviceaccount/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-leader-election-role serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-llmisvcconfig-read-access serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/llmisvc-leader-election-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-admin serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-edit serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-inferenceservice-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-metrics-reader-cluster-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-admin serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-edit serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-view serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-proxy-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-view serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-inferenceservice-image-volume-scc serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-llmisvcconfig-read-access serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/llmisvc-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-inferenceservice-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-llmisvc-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-manager-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-proxy-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/llmisvc-manager-rolebinding serverside-applied [e2e-llm-inference-service] configmap/inferenceservice-config serverside-applied [e2e-llm-inference-service] configmap/kserve-parameters serverside-applied [e2e-llm-inference-service] secret/kserve-webhook-server-secret serverside-applied [e2e-llm-inference-service] secret/mlpipeline-s3-artifact serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-metrics-service serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-service serverside-applied [e2e-llm-inference-service] service/kserve-webhook-server-service serverside-applied [e2e-llm-inference-service] service/llmisvc-controller-manager-service serverside-applied [e2e-llm-inference-service] service/llmisvc-webhook-server-service serverside-applied [e2e-llm-inference-service] service/s3-service serverside-applied [e2e-llm-inference-service] deployment.apps/kserve-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/seaweedfs serverside-applied [e2e-llm-inference-service] networkpolicy.networking.k8s.io/kserve-controller-manager serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-inferenceservice-image-volume-scc serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] clusterstoragecontainer.serving.kserve.io/default serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-router-route serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-scheduler serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-tracing serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-worker-data-parallel serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferencegraph.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceserviceconfig.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/servingruntime.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/trainedmodel.serving.kserve.io serverside-applied [e2e-llm-inference-service] Waiting for llmisvc-controller-manager to be ready... [e2e-llm-inference-service] Waiting for pod -l "control-plane=llmisvc-controller-manager" in namespace "kserve" to be created... [e2e-llm-inference-service] Pod -l "control-plane=llmisvc-controller-manager" in namespace "kserve" found. [e2e-llm-inference-service] Current pods for -l "control-plane=llmisvc-controller-manager" in namespace "kserve": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] llmisvc-controller-manager-d4c968599-926ss 0/1 Running 0 6s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=llmisvc-controller-manager" in namespace "kserve" to become ready... [e2e-llm-inference-service] pod/llmisvc-controller-manager-d4c968599-926ss condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=llmisvc-controller-manager" in namespace "kserve" are ready. [e2e-llm-inference-service] Re-Applying all resources... [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencemodelrewrites.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceobjectives.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepoolimports.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied [e2e-llm-inference-service] serviceaccount/kserve-controller-manager serverside-applied [e2e-llm-inference-service] serviceaccount/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-leader-election-role serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-llmisvcconfig-read-access serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/llmisvc-leader-election-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-admin serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-edit serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-inferenceservice-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-metrics-reader-cluster-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-admin serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-edit serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-view serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-proxy-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-view serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-inferenceservice-image-volume-scc serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-llmisvcconfig-read-access serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/llmisvc-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-inferenceservice-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-llmisvc-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-manager-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-proxy-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/llmisvc-manager-rolebinding serverside-applied [e2e-llm-inference-service] configmap/inferenceservice-config serverside-applied [e2e-llm-inference-service] configmap/kserve-parameters serverside-applied [e2e-llm-inference-service] secret/kserve-webhook-server-secret serverside-applied [e2e-llm-inference-service] secret/mlpipeline-s3-artifact serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-metrics-service serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-service serverside-applied [e2e-llm-inference-service] service/kserve-webhook-server-service serverside-applied [e2e-llm-inference-service] service/llmisvc-controller-manager-service serverside-applied [e2e-llm-inference-service] service/llmisvc-webhook-server-service serverside-applied [e2e-llm-inference-service] service/s3-service serverside-applied [e2e-llm-inference-service] deployment.apps/kserve-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/seaweedfs serverside-applied [e2e-llm-inference-service] networkpolicy.networking.k8s.io/kserve-controller-manager serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-inferenceservice-image-volume-scc serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] clusterstoragecontainer.serving.kserve.io/default serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-router-route serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-scheduler serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-tracing serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-worker-data-parallel serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferencegraph.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceserviceconfig.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/servingruntime.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/trainedmodel.serving.kserve.io serverside-applied [e2e-llm-inference-service] Applying DSC/DSCI resources... [e2e-llm-inference-service] dscinitialization.dscinitialization.opendatahub.io/test-dsci created [e2e-llm-inference-service] datasciencecluster.datasciencecluster.opendatahub.io/test-dsc created [e2e-llm-inference-service] KServe manual installation complete [e2e-llm-inference-service] 🔧 Configuration: [e2e-llm-inference-service] KServe deployment: ❌ disabled [e2e-llm-inference-service] Kuadrant deployment: ✅ enabled [e2e-llm-inference-service] [e2e-llm-inference-service] Checking OpenShift server version...(4.21.24) [e2e-llm-inference-service] 🎯 Server version (4.21.24) is 4.19.9 or higher - continue with the script [e2e-llm-inference-service] ⏳ Installing cert-manager [e2e-llm-inference-service] namespace/cert-manager-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/openshift-cert-manager-operator created [e2e-llm-inference-service] subscription.operators.coreos.com/openshift-cert-manager-operator created [e2e-llm-inference-service] Waiting for openshift-cert-manager-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (0/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (5/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (10/300) [e2e-llm-inference-service] CSV cert-manager-operator.v1.20.0 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD certificates.cert-manager.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD certificates.cert-manager.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io condition met [e2e-llm-inference-service] ✅ Cert-manager installed [e2e-llm-inference-service] ⏳ Installing openshift-lws-operator [e2e-llm-inference-service] namespace/openshift-lws-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/leader-worker-set created [e2e-llm-inference-service] subscription.operators.coreos.com/leader-worker-set created [e2e-llm-inference-service] Waiting for leader-worker-set CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (0/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (5/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (10/300) [e2e-llm-inference-service] CSV leader-worker-set.v1.0.0 found, but not yet Succeeded (Phase: Installing). Waiting... (15/300) [e2e-llm-inference-service] CSV leader-worker-set.v1.0.0 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD leaderworkersetoperators.operator.openshift.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD leaderworkersetoperators.operator.openshift.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/leaderworkersetoperators.operator.openshift.io condition met [e2e-llm-inference-service] leaderworkersetoperator.operator.openshift.io/cluster created [e2e-llm-inference-service] ⏳ waiting for openshift-lws-operator to be ready.… [e2e-llm-inference-service] Waiting for pod -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" to be created... [e2e-llm-inference-service] Pod -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" found. [e2e-llm-inference-service] Current pods for -l "name=openshift-lws-operator" in namespace "openshift-lws-operator": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] openshift-lws-operator-fd8ccff4c-2hxnc 1/1 Running 0 13s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" to become ready... [e2e-llm-inference-service] pod/openshift-lws-operator-fd8ccff4c-2hxnc condition met [e2e-llm-inference-service] Pod(s) -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" are ready. [e2e-llm-inference-service] ✅ openshift-lws-operator installed [e2e-llm-inference-service] gatewayclass.gateway.networking.k8s.io/openshift-default created [e2e-llm-inference-service] Waiting for pod -l "app=istiod" in namespace "openshift-ingress" to be created... [e2e-llm-inference-service] Pod -l "app=istiod" in namespace "openshift-ingress" found. [e2e-llm-inference-service] Current pods for -l "app=istiod" in namespace "openshift-ingress": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] istiod-openshift-gateway-94bb8fbfd-wnhkc 0/1 Running 0 5s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "app=istiod" in namespace "openshift-ingress" to become ready... [e2e-llm-inference-service] pod/istiod-openshift-gateway-94bb8fbfd-wnhkc condition met [e2e-llm-inference-service] Pod(s) -l "app=istiod" in namespace "openshift-ingress" are ready. [e2e-llm-inference-service] ⏳ Creating a Gateway [e2e-llm-inference-service] Error from server (AlreadyExists): namespaces "openshift-ingress" already exists [e2e-llm-inference-service] ⏳ Creating gateway memory ConfigMap for parametersRef (2Gi) [e2e-llm-inference-service] configmap/gateway-proxy-config created [e2e-llm-inference-service] gateway.gateway.networking.k8s.io/openshift-ai-inference created [e2e-llm-inference-service] Waiting for pod -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" to be created... [e2e-llm-inference-service] Pod -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" found. [e2e-llm-inference-service] Current pods for -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] openshift-ai-inference-openshift-default-866675cbb8-qnxtj 0/1 Running 0 6s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" to become ready... [e2e-llm-inference-service] pod/openshift-ai-inference-openshift-default-866675cbb8-qnxtj condition met [e2e-llm-inference-service] Pod(s) -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" are ready. [e2e-llm-inference-service] ⏳ Installing RHCL(Kuadrant) operator [e2e-llm-inference-service] namespace/kuadrant-system created [e2e-llm-inference-service] subscription.operators.coreos.com/rhcl-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/kuadrant created [e2e-llm-inference-service] Waiting for rhcl-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (0/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (5/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (10/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (15/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (20/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Installing). Waiting... (25/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Installing). Waiting... (30/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Installing). Waiting... (35/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD kuadrants.kuadrant.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD kuadrants.kuadrant.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/kuadrants.kuadrant.io condition met [e2e-llm-inference-service] Waiting for apiserver discovery /apis/kuadrant.io/v1beta1 to list kuadrants (timeout: 120s)… [e2e-llm-inference-service] Discovery for kuadrant.io/v1beta1 includes kuadrants. [e2e-llm-inference-service] ⏳ sleeping 30s after discovery (RESTMapper can trail discovery)… [e2e-llm-inference-service] kuadrant.kuadrant.io/kuadrant created [e2e-llm-inference-service] ⏳ waiting for Kuadrant Ready (attempt 1/2, timeout 5m)… [e2e-llm-inference-service] kuadrant.kuadrant.io/kuadrant condition met [e2e-llm-inference-service] Waiting for pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" to be created... [e2e-llm-inference-service] Pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" found. [e2e-llm-inference-service] Current pods for -l "control-plane=authorino-operator" in namespace "kuadrant-system": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] authorino-operator-6d85f6564-5q86f 1/1 Running 0 68s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" to become ready... [e2e-llm-inference-service] pod/authorino-operator-6d85f6564-5q86f condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" are ready. [e2e-llm-inference-service] ⏳ waiting for authorino service to be created... [e2e-llm-inference-service] service/authorino-authorino-authorization condition met [e2e-llm-inference-service] service/authorino-authorino-authorization annotated [e2e-llm-inference-service] Warning: resource authorinos/authorino is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically. [e2e-llm-inference-service] authorino.operator.authorino.kuadrant.io/authorino configured [e2e-llm-inference-service] Waiting for pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" to be created... [e2e-llm-inference-service] Pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" found. [e2e-llm-inference-service] Current pods for -l "control-plane=authorino-operator" in namespace "kuadrant-system": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] authorino-operator-6d85f6564-5q86f 1/1 Running 0 78s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" to become ready... [e2e-llm-inference-service] pod/authorino-operator-6d85f6564-5q86f condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" are ready. [e2e-llm-inference-service] ✅ kuadrant(authorino) installed [e2e-llm-inference-service] Patching ingress domain... [e2e-llm-inference-service] configmap/inferenceservice-config patched [e2e-llm-inference-service] pod "kserve-controller-manager-576d7b7757-ss7x8" deleted [e2e-llm-inference-service] Waiting for kserve-controller-manager to be ready... [e2e-llm-inference-service] pod/kserve-controller-manager-576d7b7757-pddx2 condition met [e2e-llm-inference-service] Installing ODH Model Controller manually... [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/accounts.nim.opendatahub.io created [e2e-llm-inference-service] serviceaccount/model-serving-api created [e2e-llm-inference-service] serviceaccount/odh-model-controller created [e2e-llm-inference-service] role.rbac.authorization.k8s.io/leader-election-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/account-editor-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/account-viewer-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-prometheus-k8s created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/metrics-reader created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/model-serving-api created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/odh-model-controller-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/proxy-role created [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/leader-election-rolebinding created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/model-serving-api created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/odh-model-controller-rolebinding-opendatahub created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/proxy-rolebinding created [e2e-llm-inference-service] configmap/odh-model-controller-parameters created [e2e-llm-inference-service] service/model-serving-api created [e2e-llm-inference-service] service/odh-model-controller-metrics-service created [e2e-llm-inference-service] service/odh-model-controller-webhook-service created [e2e-llm-inference-service] deployment.apps/model-serving-api created [e2e-llm-inference-service] deployment.apps/odh-model-controller created [e2e-llm-inference-service] servicemonitor.monitoring.coreos.com/model-serving-api-metrics created [e2e-llm-inference-service] servicemonitor.monitoring.coreos.com/odh-model-controller-metrics-monitor created [e2e-llm-inference-service] template.template.openshift.io/autogluon-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/guardrails-detector-huggingface-serving-template created [e2e-llm-inference-service] template.template.openshift.io/kserve-ovms created [e2e-llm-inference-service] template.template.openshift.io/mlserver-cuda-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/mlserver-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-x86-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-x86-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-x86-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cuda-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cuda-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cuda-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-gaudi-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-gaudi-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-gaudi-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-multinode-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-multinode-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-multinode-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-rocm-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-rocm-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-rocm-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-ppc64le-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-ppc64le-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-ppc64le-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-s390x-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-s390x-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-s390x-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-x86-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-x86-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-x86-runtime-template-fast-2 created [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/mutating.odh-model-controller.opendatahub.io created [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/validating.odh-model-controller.opendatahub.io created [e2e-llm-inference-service] Waiting for deployment "odh-model-controller" rollout to finish: 0 of 1 updated replicas are available... [e2e-llm-inference-service] deployment "odh-model-controller" successfully rolled out [e2e-llm-inference-service] networkpolicy.networking.k8s.io/allow-all created [e2e-llm-inference-service] KServe setup complete (namespace: kserve) [e2e-llm-inference-service] Add testing models to SeaweedFS S3 storage ... [e2e-llm-inference-service] Waiting for SeaweedFS deployment to be ready... [e2e-llm-inference-service] deployment "seaweedfs" successfully rolled out [e2e-llm-inference-service] S3 init job not completed, re-creating... [e2e-llm-inference-service] job.batch/s3-init replaced [e2e-llm-inference-service] Waiting for S3 init job to complete... [e2e-llm-inference-service] job.batch/s3-init condition met [e2e-llm-inference-service] Prepare CI namespace and install ServingRuntimes [e2e-llm-inference-service] Setting up CI namespace: kserve-ci-e2e-test [e2e-llm-inference-service] Tearing down CI namespace: kserve-ci-e2e-test [e2e-llm-inference-service] Namespace kserve-ci-e2e-test does not exist, skipping deletion [e2e-llm-inference-service] CI namespace teardown complete [e2e-llm-inference-service] Creating namespace kserve-ci-e2e-test [e2e-llm-inference-service] namespace/kserve-ci-e2e-test created [e2e-llm-inference-service] Applying S3 artifact secret [e2e-llm-inference-service] secret/mlpipeline-s3-artifact created [e2e-llm-inference-service] Applying storage-config secret [e2e-llm-inference-service] secret/storage-config created [e2e-llm-inference-service] Applying SeaweedFS S3 credentials secret [e2e-llm-inference-service] secret/seaweedfs-s3-creds created [e2e-llm-inference-service] Linking seaweedfs-s3-creds to default service account [e2e-llm-inference-service] Creating odh-trusted-ca-bundle configmap [e2e-llm-inference-service] configmap/odh-trusted-ca-bundle created [e2e-llm-inference-service] Installing ServingRuntimes [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-autogluonserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-huggingfaceserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-huggingfaceserver-multinode created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-lgbserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-mlserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-paddleserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-pmmlserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-predictiveserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-sklearnserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-tensorflow-serving created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-torchserve created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-tritonserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-vllmserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-xgbserver created [e2e-llm-inference-service] CI namespace setup complete [e2e-llm-inference-service] Setup complete [e2e-llm-inference-service] === E2E cluster / operator summary === [e2e-llm-inference-service] Client Version: 4.20.11 [e2e-llm-inference-service] Kustomize Version: v5.6.0 [e2e-llm-inference-service] Server Version: 4.21.24 [e2e-llm-inference-service] Kubernetes Version: v1.34.9 [e2e-llm-inference-service] ClusterVersion desired: 4.21.24 [e2e-llm-inference-service] ClusterVersion history (latest): 4.21.24 (Completed) [e2e-llm-inference-service] CSVs in kuadrant-system: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in openshift-keda: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] custom-metrics-autoscaler.v2.19.0-2 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in cert-manager-operator: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in openshift-lws-operator: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] leader-worker-set.v1.0.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in openshift-operators (ODH / shared operators, filtered): [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] Kuadrant / Authorino (diagnostics): [e2e-llm-inference-service] CRD kuadrants.kuadrant.io versions: v1beta1 served=true storage=true [e2e-llm-inference-service] Subscriptions in kuadrant-system: [e2e-llm-inference-service] authorino-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators authorino-operator.v1.4.1 [e2e-llm-inference-service] dns-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators dns-operator.v1.4.0 [e2e-llm-inference-service] limitador-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators limitador-operator.v1.4.0 [e2e-llm-inference-service] rhcl-operator stable redhat-operators rhcl-operator.v1.4.1 [e2e-llm-inference-service] Kuadrant CR conditions (kuadrant/kuadrant-system): [e2e-llm-inference-service] Ready=True (Ready) [e2e-llm-inference-service] KServe deployments in kserve: [e2e-llm-inference-service] kserve-controller-manager: ready=1 image=quay.io/opendatahub/kserve-controller@sha256:7912d58f195183d9e721116618865eb65dab4d9d594a55ba497d4966a5572fda [e2e-llm-inference-service] imageID: quay.io/opendatahub/kserve-controller@sha256:09bf247709b9e97d030f2f55a0dd7e2382e6c1731c4414ca9a1f899e14e574c4 [e2e-llm-inference-service] odh-model-controller: ready=1 image=quay.io/opendatahub/odh-model-controller:fast [e2e-llm-inference-service] imageID: quay.io/opendatahub/odh-model-controller@sha256:4d2ce35df65bab6d2693c9c94b3f24844e28d1e34eb5c52b243f168e1bce3e01 [e2e-llm-inference-service] llmisvc-controller-manager: ready=1 image=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:d69df2f43218d8e9f989e3e2daf80e41a50953a20f5e2534497cb1de5b055298 [e2e-llm-inference-service] imageID: quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:837a679e32b1c64b4c792f1949efdd3a517a0f0f0caf716b1602f435ab266503 [e2e-llm-inference-service] === End E2E cluster / operator summary === [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] CA certificate extracted [e2e-llm-inference-service] REQUESTS_CA_BUNDLE=/tmp/ca.crt [e2e-llm-inference-service] Run E2E tests: llminferenceservice and cluster_cpu and not autoscaling and not tracing [e2e-llm-inference-service] Starting E2E functional tests ... [e2e-llm-inference-service] Parallelism requested for pytest is 2 [e2e-llm-inference-service] ============================= test session starts ============================== [e2e-llm-inference-service] platform linux -- Python 3.11.13, pytest-7.4.4, pluggy-1.5.0 -- /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] cachedir: .pytest_cache [e2e-llm-inference-service] metadata: {'Python': '3.11.13', 'Platform': 'Linux-5.14.0-570.122.1.el9_6.x86_64-x86_64-with-glibc2.34', 'Packages': {'pytest': '7.4.4', 'pluggy': '1.5.0'}, 'Plugins': {'metadata': '3.1.1', 'anyio': '4.9.0', 'asyncio': '0.23.8', 'xdist': '3.6.1', 'httpx': '0.30.0', 'json-report': '1.5.0', 'cov': '5.0.0'}, 'PLATFORM': 'el9'} [e2e-llm-inference-service] rootdir: /workspace/source/test/e2e [e2e-llm-inference-service] configfile: pytest.ini [e2e-llm-inference-service] plugins: metadata-3.1.1, anyio-4.9.0, asyncio-0.23.8, xdist-3.6.1, httpx-0.30.0, json-report-1.5.0, cov-5.0.0 [e2e-llm-inference-service] asyncio: mode=Mode.STRICT [e2e-llm-inference-service] created: 2/2 workers [e2e-llm-inference-service] 2 workers [42 items] [e2e-llm-inference-service] [e2e-llm-inference-service] scheduling tests via WorkStealingScheduling [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-with-section-name] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] 2026-07-16 17:48:16.535 6175 kserve INFO [conftest.py:configure_logger():40] Logger configured [e2e-llm-inference-service] 2026-07-16 17:48:16.535 6172 kserve INFO [conftest.py:configure_logger():40] Logger configured [e2e-llm-inference-service] 2026-07-16 17:48:16.603 6172 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 17:48:16.603 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 17:48:16.632 6172 kserve.trace Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-16 17:48:16.632 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():62] Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-16 17:48:16.641 6172 kserve.trace ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-16 17:48:16.641 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():70] ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-with-section-name] [e2e-llm-inference-service] llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-without-section-name] 2026-07-16 17:48:27.601 6172 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 17:48:27.601 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 17:48:27.645 6172 kserve.trace ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-16 17:48:27.645 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():57] ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-without-section-name] [e2e-llm-inference-service] llmisvc/test_llm_auth.py::test_llm_auth_enabled_requires_token[cluster_cpu-cluster_single_node-auth-enabled-default] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator0] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator1] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_auth.py::test_llm_auth_enabled_requires_token[cluster_cpu-cluster_single_node-auth-enabled-default] [e2e-llm-inference-service] llmisvc/test_llm_auth.py::test_llm_auth_invalid_token_rejected[cluster_cpu-cluster_single_node-auth-invalid-token] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator2] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_auth.py::test_llm_auth_invalid_token_rejected[cluster_cpu-cluster_single_node-auth-invalid-token] [e2e-llm-inference-service] llmisvc/test_llm_auth.py::test_llm_auth_disabled_no_token_required[cluster_cpu-cluster_single_node-auth-disabled] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator2] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf0] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_auth.py::test_llm_auth_disabled_no_token_required[cluster_cpu-cluster_single_node-auth-disabled] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] 2026-07-16 17:57:46.857 6172 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 17:57:46.857 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 17:57:46.906 6172 kserve.trace ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-16 17:57:46.906 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():57] ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf1] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-pvc] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-pvc] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-pvc] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] 2026-07-16 18:09:12.561 6172 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:09:12.561 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:09:12.611 6172 kserve.trace ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-16 18:09:12.611 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():57] ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-16 18:09:12.612 6172 kserve.trace Checking HttpRoute router-route-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:09:12.612 6172 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:09:12.616 6172 kserve.trace Resource not found, creating HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-16 18:09:12.616 6172 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-16 18:09:12.625 6172 kserve.trace ✓ Successfully created HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-16 18:09:12.625 6172 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-16 18:09:12.625 6172 kserve.trace Checking HttpRoute router-route-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:09:12.625 6172 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:09:12.629 6172 kserve.trace Resource not found, creating HttpRoute router-route-2 [e2e-llm-inference-service] 2026-07-16 18:09:12.629 6172 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-2 [e2e-llm-inference-service] 2026-07-16 18:09:12.637 6172 kserve.trace ✓ Successfully created HttpRoute router-route-2 [e2e-llm-inference-service] 2026-07-16 18:09:12.637 6172 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-2 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha1_to_v1alpha2_conversion [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha1_to_v1alpha2_conversion [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha2_to_v1alpha1_conversion [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha2_to_v1alpha1_conversion [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_criticality_preservation_via_annotations [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_criticality_preservation_via_annotations [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_lora_criticality_preservation [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_lora_criticality_preservation [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_round_trip_conversion_preserves_fields [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_round_trip_conversion_preserves_fields [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-single-lora-adapter-hf] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] 2026-07-16 18:20:18.001 6172 kserve.trace Checking Gateway router-gateway-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:20:18.001 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:20:18.039 6172 kserve.trace Resource not found, creating Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-16 18:20:18.039 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():62] Resource not found, creating Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-16 18:20:18.048 6172 kserve.trace ✓ Successfully created Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-16 18:20:18.048 6172 kserve.trace INFO [gw_api.py:create_or_update_gateway():70] ✓ Successfully created Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-16 18:20:18.049 6172 kserve.trace Checking HttpRoute router-route-3 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:20:18.049 6172 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-3 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:20:18.053 6172 kserve.trace Resource not found, creating HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-16 18:20:18.053 6172 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-16 18:20:18.065 6172 kserve.trace ✓ Successfully created HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-16 18:20:18.065 6172 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-16 18:20:18.065 6172 kserve.trace Checking HttpRoute router-route-4 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:20:18.065 6172 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-4 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-16 18:20:18.070 6172 kserve.trace Resource not found, creating HttpRoute router-route-4 [e2e-llm-inference-service] 2026-07-16 18:20:18.070 6172 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-4 [e2e-llm-inference-service] 2026-07-16 18:20:18.082 6172 kserve.trace ✓ Successfully created HttpRoute router-route-4 [e2e-llm-inference-service] 2026-07-16 18:20:18.082 6172 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-4 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-single-lora-adapter-hf] [e2e-llm-inference-service] llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-multiple-lora-adapters] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-multiple-lora-adapters] [e2e-llm-inference-service] llmisvc/test_llm_tls.py::test_llm_tls_resources[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_tls.py::test_llm_tls_resources[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_prestop_hook.py::test_prestop_hook[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_prestop_hook.py::test_prestop_hook[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_rolling_upgrade.py::test_rolling_upgrade_coordination[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-qwen2.5-0.5b] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_rolling_upgrade.py::test_rolling_upgrade_coordination[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_storage_version_migration.py::TestStorageVersionMigration::test_storage_version_migration_after_simulated_upgrade [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-qwen2.5-0.5b] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_storage_version_migration.py::TestStorageVersionMigration::test_storage_version_migration_after_simulated_upgrade [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-custom-template-workload-llmd-simulator] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-custom-template-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-nonzero-threshold-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-nonzero-threshold-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] [gw1] FAILED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] [e2e-llm-inference-service] =================================== FAILURES =================================== [e2e-llm-inference-service] _ test_llm_inference_service[router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] _ [e2e-llm-inference-service] [gw1] linux -- Python 3.11.13 /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'scheduler-v06-pd-config-migration', 'workload-llmd-simulator-pd'], prompt='KSer... {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] @pytest.mark.parametrize( [e2e-llm-inference-service] "test_case", [e2e-llm-inference-service] [ [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-gateway-ref", [e2e-llm-inference-service] "router-with-managed-route", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/completions", [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=create_response_assertion(with_field="choices"), [e2e-llm-inference-service] expected_gateway=ROUTER_GATEWAYS[0], [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[0]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-custom-route-timeout", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="custom-route-timeout-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-refs", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="router-with-refs-test", [e2e-llm-inference-service] expected_gateway=ROUTER_GATEWAYS[0], [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[0]], [e2e-llm-inference-service] routes=[ROUTER_ROUTES[0], ROUTER_ROUTES[1]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=["router-managed", "workload-pd-cpu", "model-fb-opt-125m"], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-custom-route-timeout-pd", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-pd-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] service_name="custom-route-timeout-pd-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-refs-pd", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-pd-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] service_name="router-with-refs-pd-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] expected_gateway=ROUTER_GATEWAYS[1], [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[1]], [e2e-llm-inference-service] routes=[ROUTER_ROUTES[2], ROUTER_ROUTES[3]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-dp-ep-gpu", [e2e-llm-inference-service] "workload-dp-ep-prefill-gpu", [e2e-llm-inference-service] "model-deepseek-v2-lite", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="Delve into the multifaceted implications of a fully disaggregated cloud architecture, specifically " [e2e-llm-inference-service] "where the compute plane (P) and the data plane (D) are independently deployed and managed for a " [e2e-llm-inference-service] "geographically distributed, high-throughput, low-latency microservices ecosystem. Beyond the " [e2e-llm-inference-service] "fundamental challenges of network latency and data consistency, elaborate on the advanced " [e2e-llm-inference-service] "considerations and trade-offs inherent in such a setup: 1. Network Architecture and Protocols: " [e2e-llm-inference-service] "How would the network fabric and underlying protocols (e.g., RDMA, custom transport layers) need to " [e2e-llm-inference-service] "evolve to support optimal performance and minimize inter-plane communication overhead, especially for " [e2e-llm-inference-service] "synchronous operations? Discuss the role of network programmability (e.g., SDN, P4) in dynamically " [e2e-llm-inference-service] "optimizing routing and traffic flow between P and D. 2. Advanced Data Consistency and Durability: " [e2e-llm-inference-service] "Explore sophisticated data consistency models (e.g., causal consistency, strong eventual consistency) " [e2e-llm-inference-service] "and their applicability in balancing performance and data integrity across a globally distributed data plane. " [e2e-llm-inference-service] "Detail strategies for ensuring data durability and fault tolerance, including multi-region replication, " [e2e-llm-inference-service] "intelligent partitioning, and recovery mechanisms in the event of partial or full plane failures. " [e2e-llm-inference-service] "3. Dynamic Resource Orchestration and Cost Optimization: Analyze how an orchestration layer would intelligently " [e2e-llm-inference-service] "manage the independent scaling of compute (P) and data (D) resources, considering fluctuating workloads, " [e2e-llm-inference-service] "cost efficiency, and performance targets (e.g., using predictive analytics for resource provisioning). " [e2e-llm-inference-service] "Discuss mechanisms for dynamically reallocating compute nodes to different data partitions based on " [e2e-llm-inference-service] "workload patterns and data locality, potentially involving live migration strategies. " [e2e-llm-inference-service] "4. Security and Compliance in a Distributed Landscape: Address the enhanced security perimeter " [e2e-llm-inference-service] "challenges, including securing communication channels between P and D (encryption in transit, mutual TLS), " [e2e-llm-inference-service] "fine-grained access control to data at rest and in motion, and identity management across disaggregated " [e2e-llm-inference-service] "components. Discuss how such an architecture impacts compliance with regulatory frameworks (e.g., GDPR, HIPAA) " [e2e-llm-inference-service] "concerning data sovereignty, privacy, and auditability. 5. Operational Complexity and Observability: " [e2e-llm-inference-service] "Examine the increased complexity in monitoring, logging, and tracing across highly decoupled compute and " [e2e-llm-inference-service] "data planes. What specialized tooling and practices (e.g., distributed tracing with OpenTelemetry, advanced AIOps) " [e2e-llm-inference-service] "would be essential? How would incident response and troubleshooting differ in this disaggregated environment " [e2e-llm-inference-service] "compared to traditional integrated systems? Consider the challenges of pinpointing root causes across " [e2e-llm-inference-service] "independent failures. 6. Real-world Applicability and Future Trends: Identify specific industries " [e2e-llm-inference-service] "or use cases (e.g., high-frequency trading, IoT edge processing, large language model inference) " [e2e-llm-inference-service] "where the benefits of P/D disaggregation would strongly outweigh its complexities. " [e2e-llm-inference-service] "Conclude by speculating on emerging technologies or paradigms (e.g., serverless compute functions " [e2e-llm-inference-service] "directly interacting with object storage, in-memory disaggregation) that could further drive or " [e2e-llm-inference-service] "transform P/D disaggregation in cloud computing.", [e2e-llm-inference-service] max_tokens=2000, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_gpu, [e2e-llm-inference-service] pytest.mark.cluster_nvidia, [e2e-llm-inference-service] pytest.mark.cluster_nvidia_roce, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-no-scheduler", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="What is KServe?", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.no_scheduler, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-simulated-dp-ep-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="This test simulates DP+EP that can run on CPU, the idea is to test the LWS-based deployment, " [e2e-llm-inference-service] "but without the resources requirements for DP+EP (GPUs and ROCe/IB).", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_multi_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Scheduler config tests [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-inline-config", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-inline-config-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Chat completions endpoint coverage [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] "model-qwen2.5-0.5b", [e2e-llm-inference-service] ], [e2e-llm-inference-service] model_name="Qwen/Qwen2.5-0.5B-Instruct", [e2e-llm-inference-service] endpoint="/v1/chat/completions", [e2e-llm-inference-service] prompt="What is KServe?", [e2e-llm-inference-service] payload_formatter=chat_completions_payload, [e2e-llm-inference-service] response_assertion=create_response_assertion(with_field="choices"), [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-configmap-ref", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-configmap-ref-test", [e2e-llm-inference-service] before_test=[create_scheduler_configmap], [e2e-llm-inference-service] after_test=[delete_scheduler_configmap], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-replicas", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-ha-replicas-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-custom-template", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-custom-template-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Scheduler v0.6 → v0.7 migration tests. [e2e-llm-inference-service] # Deploy v0.6-style configs and verify the controller migrates them [e2e-llm-inference-service] # so the v0.7 scheduler boots successfully. [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-v06-pd-config-migration", [e2e-llm-inference-service] "workload-llmd-simulator-pd", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-v06-pd-migration-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-v06-nonzero-threshold-migration", [e2e-llm-inference-service] "workload-llmd-simulator-pd", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-v06-threshold-migration-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Precise prefix KV cache routing test [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-precise-prefix-cache-inline-config", [e2e-llm-inference-service] "workload-llmd-simulator-kvcache", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="precise-prefix-cache-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Models endpoint coverage [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/models", [e2e-llm-inference-service] response_assertion=create_response_assertion(with_field="data"), [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Model-based routing via X-Gateway-Model-Name header — /v1/completions [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/completions", [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=assert_model_field_matches("facebook/opt-125m"), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/facebook/opt-125m", [e2e-llm-inference-service] }, [e2e-llm-inference-service] peers=[ [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] "model-qwen2.5-0.5b", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/completions", [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=assert_model_field_matches( [e2e-llm-inference-service] "Qwen/Qwen2.5-0.5B-Instruct" [e2e-llm-inference-service] ), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/Qwen/Qwen2.5-0.5B-Instruct", [e2e-llm-inference-service] }, [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] pytest.mark.model_routing, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Model-based routing via X-Gateway-Model-Name header — /v1/chat/completions [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/chat/completions", [e2e-llm-inference-service] prompt="What is KServe?", [e2e-llm-inference-service] payload_formatter=chat_completions_payload, [e2e-llm-inference-service] response_assertion=assert_model_field_matches("facebook/opt-125m"), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/facebook/opt-125m", [e2e-llm-inference-service] }, [e2e-llm-inference-service] peers=[ [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] "model-qwen2.5-0.5b", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/chat/completions", [e2e-llm-inference-service] prompt="What is KServe?", [e2e-llm-inference-service] payload_formatter=chat_completions_payload, [e2e-llm-inference-service] response_assertion=assert_model_field_matches( [e2e-llm-inference-service] "Qwen/Qwen2.5-0.5B-Instruct" [e2e-llm-inference-service] ), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/Qwen/Qwen2.5-0.5B-Instruct", [e2e-llm-inference-service] }, [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] pytest.mark.model_routing, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Model-based routing via X-Gateway-Model-Name header — LoRA adapter [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m-with-lora-hf", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/completions", [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] model_name=f"publishers/{KSERVE_TEST_NAMESPACE}/models/lora-adapter-1", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=assert_model_field_matches( [e2e-llm-inference-service] f"publishers/{KSERVE_TEST_NAMESPACE}/models/lora-adapter-1" [e2e-llm-inference-service] ), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/lora-adapter-1", [e2e-llm-inference-service] }, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.model_routing, [e2e-llm-inference-service] pytest.mark.lora, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Model-based routing via X-Gateway-Model-Name header — /v1/models (base + LoRA) [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m-with-lora-hf", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/models", [e2e-llm-inference-service] response_assertion=assert_models_contains( [e2e-llm-inference-service] "facebook/opt-125m", [e2e-llm-inference-service] f"publishers/{KSERVE_TEST_NAMESPACE}/models/facebook/opt-125m", [e2e-llm-inference-service] "lora-adapter-1", [e2e-llm-inference-service] f"publishers/{KSERVE_TEST_NAMESPACE}/models/lora-adapter-1", [e2e-llm-inference-service] ), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/facebook/opt-125m", [e2e-llm-inference-service] }, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.model_routing, [e2e-llm-inference-service] pytest.mark.lora, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # PVC storage tests -- validate direct PVC volume mount with real vLLM serving [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-pvc", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] before_test=[ensure_pvc_with_model], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.pvc_storage, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-pd-cpu", [e2e-llm-inference-service] "model-pvc", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] before_test=[ensure_pvc_with_model], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.pvc_storage, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-simulated-dp-ep-cpu", [e2e-llm-inference-service] "model-pvc", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] before_test=[ensure_pvc_with_model], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_multi_node, [e2e-llm-inference-service] pytest.mark.pvc_storage, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] indirect=["test_case"], [e2e-llm-inference-service] ids=generate_test_id, [e2e-llm-inference-service] ) [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def test_llm_inference_service(test_case: TestCase): # noqa: F811 [e2e-llm-inference-service] inject_k8s_proxy() [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = KServeClient( [e2e-llm-inference-service] config_file=os.environ.get("KUBECONFIG", "~/.kube/config"), [e2e-llm-inference-service] client_configuration=client.Configuration(), [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] service_name = test_case.llm_service.metadata.name [e2e-llm-inference-service] if not test_case.llm_service.metadata.annotations: [e2e-llm-inference-service] test_case.llm_service.metadata.annotations = {} [e2e-llm-inference-service] [e2e-llm-inference-service] test_case.llm_service.metadata.annotations[ [e2e-llm-inference-service] "security.opendatahub.io/enable-auth" [e2e-llm-inference-service] ] = "false" [e2e-llm-inference-service] prefix = test_case.log_prefix [e2e-llm-inference-service] [e2e-llm-inference-service] test_failed = False [e2e-llm-inference-service] try: [e2e-llm-inference-service] print(f"{prefix} Creating LLMInferenceService {service_name}") [e2e-llm-inference-service] create_llmisvc(kserve_client, test_case.llm_service) [e2e-llm-inference-service] print(f"{prefix} Waiting for LLMInferenceService {service_name} to be ready") [e2e-llm-inference-service] wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client, test_case.llm_service, test_case.wait_timeout [e2e-llm-inference-service] ) [e2e-llm-inference-service] print(f"{prefix} Waiting for model response from {service_name}") [e2e-llm-inference-service] > wait_for_model_response( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] test_case, [e2e-llm-inference-service] test_case.wait_timeout, [e2e-llm-inference-service] extra_headers=test_case.extra_headers, [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:816: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] args = (, TestCase(base_refs=['router-managed', 'scheduler-v0... {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m'), 900) [e2e-llm-inference-service] kwargs = {'extra_headers': None}, func_name = 'wait_for_model_response' [e2e-llm-inference-service] timestamp_start = '2026-07-16T18:37:59.238678', start_time = 1784227079.2391124 [e2e-llm-inference-service] duration = 1103.2189509868622, timestamp_end = '2026-07-16T18:56:22.458066' [e2e-llm-inference-service] [e2e-llm-inference-service] @functools.wraps(func) [e2e-llm-inference-service] def wrapper(*args, **kwargs): [e2e-llm-inference-service] func_name = func.__name__ [e2e-llm-inference-service] [e2e-llm-inference-service] timestamp_start = datetime.now().isoformat() [e2e-llm-inference-service] logger.info( [e2e-llm-inference-service] f"[{func_name}] [{timestamp_start}] start - args={args}, kwargs={kwargs}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] start_time = time.time() [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > result = func(*args, **kwargs) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/logging.py:40: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'scheduler-v06-pd-config-migration', 'workload-llmd-simulator-pd'], prompt='KSer... {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] timeout_seconds = 900, extra_headers = None [e2e-llm-inference-service] [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def wait_for_model_response( [e2e-llm-inference-service] kserve_client: KServeClient, [e2e-llm-inference-service] test_case: TestCase, # noqa: F811 [e2e-llm-inference-service] timeout_seconds: int = 900, [e2e-llm-inference-service] extra_headers: Optional[Dict[str, str]] = None, [e2e-llm-inference-service] ) -> str: [e2e-llm-inference-service] def get_successful_response(): [e2e-llm-inference-service] try: [e2e-llm-inference-service] if test_case.url_getter: [e2e-llm-inference-service] service_url = test_case.url_getter(kserve_client, test_case.llm_service) [e2e-llm-inference-service] else: [e2e-llm-inference-service] service_url = get_llm_service_url(kserve_client, test_case.llm_service) [e2e-llm-inference-service] except Exception as e: [e2e-llm-inference-service] raise AssertionError(f"❌ Failed to get service URL: {e}") from e [e2e-llm-inference-service] [e2e-llm-inference-service] model_url = service_url + test_case.endpoint [e2e-llm-inference-service] [e2e-llm-inference-service] headers = {"Content-Type": "application/json"} [e2e-llm-inference-service] if extra_headers: [e2e-llm-inference-service] headers.update(extra_headers) [e2e-llm-inference-service] [e2e-llm-inference-service] if test_case.payload_formatter is not None: [e2e-llm-inference-service] test_payload = test_case.payload_formatter(test_case) [e2e-llm-inference-service] elif test_case.prompt is not None: [e2e-llm-inference-service] test_payload = { [e2e-llm-inference-service] "model": test_case.model_name [e2e-llm-inference-service] if not extra_headers or MODEL_ROUTING_HEADER not in extra_headers [e2e-llm-inference-service] else extra_headers[MODEL_ROUTING_HEADER], [e2e-llm-inference-service] "prompt": test_case.prompt, [e2e-llm-inference-service] "max_tokens": test_case.max_tokens, [e2e-llm-inference-service] } [e2e-llm-inference-service] else: [e2e-llm-inference-service] test_payload = None [e2e-llm-inference-service] [e2e-llm-inference-service] logger.info(f"Calling LLM service at {model_url} with payload {test_payload}") [e2e-llm-inference-service] try: [e2e-llm-inference-service] if test_payload is not None: [e2e-llm-inference-service] response = post_with_retry( [e2e-llm-inference-service] model_url, [e2e-llm-inference-service] headers=headers, [e2e-llm-inference-service] json_data=test_payload, [e2e-llm-inference-service] timeout=test_case.response_timeout, [e2e-llm-inference-service] ) [e2e-llm-inference-service] else: [e2e-llm-inference-service] response = get_with_retry( [e2e-llm-inference-service] model_url, [e2e-llm-inference-service] headers=headers, [e2e-llm-inference-service] timeout=test_case.response_timeout, [e2e-llm-inference-service] ) [e2e-llm-inference-service] except Exception as e: [e2e-llm-inference-service] logger.error(f"❌ Failed to call model: {e}") [e2e-llm-inference-service] raise AssertionError(f"❌ Failed to call model: {e}") from e [e2e-llm-inference-service] [e2e-llm-inference-service] logger.info(f"Model response is {response.status_code}: {response.text[:500]}") [e2e-llm-inference-service] [e2e-llm-inference-service] if 200 <= response.status_code < 300: [e2e-llm-inference-service] return response [e2e-llm-inference-service] raise AssertionError( [e2e-llm-inference-service] f"Service returned {response.status_code}: {response.text}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] > response = wait_for(get_successful_response, timeout=timeout_seconds, interval=5.0) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1119: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] assertion_fn = .get_successful_response at 0x7f91ab9653a0> [e2e-llm-inference-service] timeout = 900, interval = 5.0 [e2e-llm-inference-service] [e2e-llm-inference-service] def wait_for( [e2e-llm-inference-service] assertion_fn: Callable[[], Any], timeout: float = 5.0, interval: float = 0.1 [e2e-llm-inference-service] ) -> Any: [e2e-llm-inference-service] """Wait for the assertion to succeed within timeout.""" [e2e-llm-inference-service] deadline = time.time() + timeout [e2e-llm-inference-service] last_msg = None [e2e-llm-inference-service] while True: [e2e-llm-inference-service] try: [e2e-llm-inference-service] > return assertion_fn() [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1215: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] def get_successful_response(): [e2e-llm-inference-service] try: [e2e-llm-inference-service] if test_case.url_getter: [e2e-llm-inference-service] service_url = test_case.url_getter(kserve_client, test_case.llm_service) [e2e-llm-inference-service] else: [e2e-llm-inference-service] service_url = get_llm_service_url(kserve_client, test_case.llm_service) [e2e-llm-inference-service] except Exception as e: [e2e-llm-inference-service] raise AssertionError(f"❌ Failed to get service URL: {e}") from e [e2e-llm-inference-service] [e2e-llm-inference-service] model_url = service_url + test_case.endpoint [e2e-llm-inference-service] [e2e-llm-inference-service] headers = {"Content-Type": "application/json"} [e2e-llm-inference-service] if extra_headers: [e2e-llm-inference-service] headers.update(extra_headers) [e2e-llm-inference-service] [e2e-llm-inference-service] if test_case.payload_formatter is not None: [e2e-llm-inference-service] test_payload = test_case.payload_formatter(test_case) [e2e-llm-inference-service] elif test_case.prompt is not None: [e2e-llm-inference-service] test_payload = { [e2e-llm-inference-service] "model": test_case.model_name [e2e-llm-inference-service] if not extra_headers or MODEL_ROUTING_HEADER not in extra_headers [e2e-llm-inference-service] else extra_headers[MODEL_ROUTING_HEADER], [e2e-llm-inference-service] "prompt": test_case.prompt, [e2e-llm-inference-service] "max_tokens": test_case.max_tokens, [e2e-llm-inference-service] } [e2e-llm-inference-service] else: [e2e-llm-inference-service] test_payload = None [e2e-llm-inference-service] [e2e-llm-inference-service] logger.info(f"Calling LLM service at {model_url} with payload {test_payload}") [e2e-llm-inference-service] try: [e2e-llm-inference-service] if test_payload is not None: [e2e-llm-inference-service] response = post_with_retry( [e2e-llm-inference-service] model_url, [e2e-llm-inference-service] headers=headers, [e2e-llm-inference-service] json_data=test_payload, [e2e-llm-inference-service] timeout=test_case.response_timeout, [e2e-llm-inference-service] ) [e2e-llm-inference-service] else: [e2e-llm-inference-service] response = get_with_retry( [e2e-llm-inference-service] model_url, [e2e-llm-inference-service] headers=headers, [e2e-llm-inference-service] timeout=test_case.response_timeout, [e2e-llm-inference-service] ) [e2e-llm-inference-service] except Exception as e: [e2e-llm-inference-service] logger.error(f"❌ Failed to call model: {e}") [e2e-llm-inference-service] raise AssertionError(f"❌ Failed to call model: {e}") from e [e2e-llm-inference-service] [e2e-llm-inference-service] logger.info(f"Model response is {response.status_code}: {response.text[:500]}") [e2e-llm-inference-service] [e2e-llm-inference-service] if 200 <= response.status_code < 300: [e2e-llm-inference-service] return response [e2e-llm-inference-service] > raise AssertionError( [e2e-llm-inference-service] f"Service returned {response.status_code}: {response.text}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] E AssertionError: Service returned 502: [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1115: AssertionError [e2e-llm-inference-service] ------------------------------ Captured log setup ------------------------------ [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1637 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1586 Checking LLMInferenceServiceConfig router-managed-scheduler-v06-pd-5a5debb2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1612 Resource not found, creating LLMInferenceServiceConfig router-managed-scheduler-v06-pd-5a5debb2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1622 ✓ Successfully created LLMInferenceServiceConfig router-managed-scheduler-v06-pd-5a5debb2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1586 Checking LLMInferenceServiceConfig scheduler-v06-pd-config-migrati-bb3264a2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1612 Resource not found, creating LLMInferenceServiceConfig scheduler-v06-pd-config-migrati-bb3264a2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1622 ✓ Successfully created LLMInferenceServiceConfig scheduler-v06-pd-config-migrati-bb3264a2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1586 Checking LLMInferenceServiceConfig workload-llmd-simulator-pd-sche-64d0fc8b in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1612 Resource not found, creating LLMInferenceServiceConfig workload-llmd-simulator-pd-sche-64d0fc8b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1622 ✓ Successfully created LLMInferenceServiceConfig workload-llmd-simulator-pd-sche-64d0fc8b [e2e-llm-inference-service] ------------------------------ Captured log call ------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [test_llm_inference_service] [2026-07-16T18:37:21.831229] start - args=(), kwargs={'test_case': TestCase(base_refs=['router-managed', 'scheduler-v06-pd-config-migration', 'workload-llmd-simulator-pd'], prompt='KServe is a', service_name='scheduler-v06-pd-migration-test', endpoint='/v1/completions', max_tokens=20, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, extra_headers=None, url_getter=None, expected_gateway=None, before_test=[], after_test=[], peers=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m')} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1637 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [create_llmisvc] [2026-07-16T18:37:21.843873] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [create_llmisvc] [2026-07-16T18:37:21.934376] end - ✅ in 0.090s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_llm_isvc_ready] [2026-07-16T18:37:21.934510] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, 900), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1222 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1222 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-16T18:37:25Z', 'message': 'The following HTTPRoutes are not ready: [kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: "False" (reason "InvalidKind", message "referencing unsupported backendRef: group \\"inference.networking.x-k8s.io\\" kind \\"InferencePool\\"")]', 'reason': 'HTTPRoutesNotReady', 'severity': 'Info', 'status': 'False', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-07-16T18:37:25Z', 'message': 'Inference Pool kserve-ci-e2e-test/scheduler-v06-pd-migration-test-inference-pool exists but no Gateway controller has accepted it yet', 'reason': 'WaitingForGateway', 'severity': 'Info', 'status': 'False', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-07-16T18:37:25Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-16T18:37:25Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-07-16T18:37:25Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-16T18:37:25Z', 'message': 'The following HTTPRoutes are not ready: [kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: "False" (reason "InvalidKind", message "referencing unsupported backendRef: group \\"inference.networking.x-k8s.io\\" kind \\"InferencePool\\"")]', 'reason': 'HTTPRoutesNotReady', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-16T18:37:25Z', 'message': 'The following HTTPRoutes are not ready: [kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: "False" (reason "InvalidKind", message "referencing unsupported backendRef: group \\"inference.networking.x-k8s.io\\" kind \\"InferencePool\\"")]', 'reason': 'HTTPRoutesNotReady', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-16T18:37:25Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-07-16T18:37:25Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1222 Waiting: Missing true conditions: {'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-16T18:37:45Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-07-16T18:37:45Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-07-16T18:37:45Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-16T18:37:45Z', 'severity': 'Info', 'status': 'True', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-07-16T18:37:25Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-16T18:37:45Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-16T18:37:45Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-16T18:37:45Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-07-16T18:37:45Z', 'status': 'True', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [wait_for_llm_isvc_ready] [2026-07-16T18:37:59.238410] end - ✅ in 37.304s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_model_response] [2026-07-16T18:37:59.238678] start - args=(, TestCase(base_refs=['router-managed', 'scheduler-v06-pd-config-migration', 'workload-llmd-simulator-pd'], prompt='KServe is a', service_name='scheduler-v06-pd-migration-test', endpoint='/v1/completions', max_tokens=20, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, extra_headers=None, url_getter=None, expected_gateway=None, before_test=[], after_test=[], peers=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m'), 900), kwargs={'extra_headers': None} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llm_service_url] [2026-07-16T18:37:59.239143] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llm_service_url] [2026-07-16T18:37:59.326675] end - ✅ in 0.087s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1092 Calling LLM service at http://a00e91d4dcfd5437ba03da222bf35e1f-1399370060.us-east-1.elb.amazonaws.com/kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/completions with payload {'model': 'facebook/opt-125m', 'prompt': 'KServe is a', 'max_tokens': 20} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1111 Model response is 502: [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1222 Waiting: Service returned 502: [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llm_service_url] [2026-07-16T18:44:08.724977] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llm_service_url] [2026-07-16T18:44:08.790652] end - ✅ in 0.065s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1092 Calling LLM service at http://a00e91d4dcfd5437ba03da222bf35e1f-1399370060.us-east-1.elb.amazonaws.com/kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/completions with payload {'model': 'facebook/opt-125m', 'prompt': 'KServe is a', 'max_tokens': 20} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1111 Model response is 502: [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llm_service_url] [2026-07-16T18:50:18.096110] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llm_service_url] [2026-07-16T18:50:18.153663] end - ✅ in 0.057s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1092 Calling LLM service at http://a00e91d4dcfd5437ba03da222bf35e1f-1399370060.us-east-1.elb.amazonaws.com/kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/completions with payload {'model': 'facebook/opt-125m', 'prompt': 'KServe is a', 'max_tokens': 20} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1111 Model response is 502: [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:test_llm_inference_service.py:1219 Timed out waiting: Service returned 502: [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [wait_for_model_response] [2026-07-16T18:56:22.458066] end - ❌ 1103.219s: Service returned 502: [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:test_llm_inference_service.py:831 [router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] ❌ ERROR: Failed to call llm inference service scheduler-v06-pd-migration-test: Service returned 502: [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1240 🔍 # Diagnostics for 'scheduler-v06-pd-migration-test' in 'kserve-ci-e2e-test' [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1241 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1242 # LLMInferenceService scheduler-v06-pd-migration-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1245 apiVersion: serving.kserve.io/v1alpha1 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] security.opendatahub.io/enable-auth: 'false' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:21Z' [e2e-llm-inference-service] finalizers: [e2e-llm-inference-service] - serving.kserve.io/llmisvc-finalizer [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:security.opendatahub.io/enable-auth: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:baseRefs: {} [e2e-llm-inference-service] manager: OpenAPI-Generator [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:21Z' [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:finalizers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] v:"serving.kserve.io/llmisvc-finalizer": {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:21Z' [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:addresses: {} [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-decode-template: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-decode-worker-data-parallel: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-prefill-template: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-prefill-worker-data-parallel: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-router-route: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-scheduler: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-template: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-tracing: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-worker-data-parallel: {} [e2e-llm-inference-service] f:appliedConfigs: {} [e2e-llm-inference-service] f:conditions: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:router: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:gateways: {} [e2e-llm-inference-service] f:scheduler: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:inferencePool: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:service: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:url: {} [e2e-llm-inference-service] f:workloads: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:prefill: {} [e2e-llm-inference-service] f:primary: {} [e2e-llm-inference-service] f:scheduler: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-16T18:37:58Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] resourceVersion: '66623' [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] spec: [e2e-llm-inference-service] baseRefs: [e2e-llm-inference-service] - name: router-managed-scheduler-v06-pd-5a5debb2 [e2e-llm-inference-service] - name: scheduler-v06-pd-config-migrati-bb3264a2 [e2e-llm-inference-service] - name: workload-llmd-simulator-pd-sche-64d0fc8b [e2e-llm-inference-service] model: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uri: '' [e2e-llm-inference-service] status: [e2e-llm-inference-service] addresses: [e2e-llm-inference-service] - name: gateway-external-model-routing [e2e-llm-inference-service] url: http://a00e91d4dcfd5437ba03da222bf35e1f-1399370060.us-east-1.elb.amazonaws.com/ [e2e-llm-inference-service] - name: gateway-external [e2e-llm-inference-service] url: http://a00e91d4dcfd5437ba03da222bf35e1f-1399370060.us-east-1.elb.amazonaws.com/kserve-ci-e2e-test/scheduler-v06-pd-migration-test [e2e-llm-inference-service] - name: gateway-internal-model-routing [e2e-llm-inference-service] url: http://openshift-ai-inference-openshift-default.openshift-ingress.svc.cluster.local/ [e2e-llm-inference-service] - name: gateway-internal [e2e-llm-inference-service] url: http://openshift-ai-inference-openshift-default.openshift-ingress.svc.cluster.local/kserve-ci-e2e-test/scheduler-v06-pd-migration-test [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] serving.kserve.io/config-llm-decode-template: kserve-config-llm-decode-template [e2e-llm-inference-service] serving.kserve.io/config-llm-decode-worker-data-parallel: kserve-config-llm-decode-worker-data-parallel [e2e-llm-inference-service] serving.kserve.io/config-llm-prefill-template: kserve-config-llm-prefill-template [e2e-llm-inference-service] serving.kserve.io/config-llm-prefill-worker-data-parallel: kserve-config-llm-prefill-worker-data-parallel [e2e-llm-inference-service] serving.kserve.io/config-llm-router-route: kserve-config-llm-router-route [e2e-llm-inference-service] serving.kserve.io/config-llm-scheduler: kserve-config-llm-scheduler [e2e-llm-inference-service] serving.kserve.io/config-llm-template: kserve-config-llm-template [e2e-llm-inference-service] serving.kserve.io/config-llm-tracing: kserve-config-llm-tracing [e2e-llm-inference-service] serving.kserve.io/config-llm-worker-data-parallel: kserve-config-llm-worker-data-parallel [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:45Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: HTTPRoutesReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:45Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: InferencePoolReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:45Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: MainWorkloadReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:45Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: PrefillWorkloadReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:25Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: PresetsCombined [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:58Z' [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Ready [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:58Z' [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: RouterReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:58Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: SchedulerWorkloadReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:45Z' [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: WorkloadsReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] url: http://a00e91d4dcfd5437ba03da222bf35e1f-1399370060.us-east-1.elb.amazonaws.com/kserve-ci-e2e-test/scheduler-v06-pd-migration-test [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:44 TIME NAMESPACE SOURCE TYPE REASON MESSAGE [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:45 -------------------------------------------------------------------------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-disabled-test-kserve-869d748779-67hnj to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.50/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:20 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:20 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:20 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:06 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.50:8000/health": dial tcp 10.133.0.50:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:26 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:41 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.50:8000/health": dial tcp 10.133.0.50:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:15 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-disabled-test-kserve-869d748779-67hnj [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-5c94d555f-2jzf7 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.51/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:26 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:40 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-disabled-test-kserve-router-scheduler-5c94d555f-2jzf7 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-disabled-test-kserve-router-scheduler-5c94d555f from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:15 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-disabled-test-kserve-869d748779 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:12 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy auth-disabled-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/auth-disabled-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "auth-disabled-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/auth-disabled-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-disabled-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-disabled-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/auth-disabled-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-disabled-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/auth-disabled-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/auth-disabled-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:16 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-disabled-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:46 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-disabled-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/auth-disabled-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:46 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/auth-disabled-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:16 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [auth-disabled-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-disabled-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-enabled-test-kserve-798878c5c7-8z67f to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:18 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.47/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:21 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulling Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:48 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 27.008s (27.008s including waiting). Image size: 3531177328 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:48 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:48 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:27 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.47:8000/health": dial tcp 10.133.0.47:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:43 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-enabled-test-kserve-798878c5c7-8z67f [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-6f46874798-5nvfk to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:31 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.30/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:32 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:32 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:32 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:43 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:55 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-enabled-test-kserve-router-scheduler-6f46874798-5nvfk [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-enabled-test-kserve-router-scheduler-6f46874798 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-enabled-test-kserve-798878c5c7 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-enabled-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/auth-enabled-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-enabled-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/auth-enabled-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/auth-enabled-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-enabled-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-enabled-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:24 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/auth-enabled-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:38 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [auth-enabled-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-enabled-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-invalid-token-test-kserve-69956d8989-pnbmz to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.48/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:23 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:09 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.48:8000/health": dial tcp 10.133.0.48:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:49 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:05 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.48:8000/health": dial tcp 10.133.0.48:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:18 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-invalid-token-test-kserve-69956d8989-pnbmz [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-invalid-token-test-kserve-router-scheduler-5649968468lk857 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-5649968468lk857 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.49/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:21 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulling Pulling image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" in 2.766s (2.766s including waiting). Image size: 179512283 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:49 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:55 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-invalid-token-test-kserve-router-scheduler-5649968468 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:18 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-invalid-token-test-kserve-69956d8989 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/auth-invalid-token-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-invalid-token-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-invalid-token-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/auth-invalid-token-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-invalid-token-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/auth-invalid-token-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/auth-invalid-token-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-invalid-token-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-invalid-token-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:49 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/auth-invalid-token-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:49 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/auth-invalid-token-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:40 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [auth-invalid-token-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:49 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-invalid-token-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:06 kserve-ci-e2e-test LLMInferenceServiceController Warning UpdateFailed Failed to update status for LLMInferenceService "conv-test-criticality": Operation cannot be fulfilled on llminferenceservices.serving.kserve.io "conv-test-criticality": the object has been modified; please apply your changes to the latest version and try again [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:06 kserve-ci-e2e-test LLMInferenceServiceController Warning UpdateFailed Failed to update status for LLMInferenceService "conv-test-lora-crit": llminferenceservices.serving.kserve.io "conv-test-lora-crit" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:07 kserve-ci-e2e-test LLMInferenceServiceController Warning UpdateFailed Failed to update status for LLMInferenceService "conv-test-round-trip": llminferenceservices.serving.kserve.io "conv-test-round-trip" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-5d9b9b4785-cb2v6 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.66/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:38 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.66:8001/health": dial tcp 10.133.0.66:8001: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:57 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:57 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: custom-route-timeout-pd-test-kserve-5d9b9b4785-cb2v6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-589f5b6bd7-9mmxn to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.67/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.67:8000/health": dial tcp 10.133.0.67:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:57 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: custom-route-timeout-pd-test-kserve-prefill-589f5b6bd7-9mmxn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set custom-route-timeout-pd-test-kserve-prefill-589f5b6bd7 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: custom-route-timeout-pd-test-kserve-router-scheduler-548f4klq5x [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-548f4klq5x to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.48/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:19 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:19 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:19 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:57 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:11 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:18 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set custom-route-timeout-pd-test-kserve-router-scheduler-548f47ccd8 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set custom-route-timeout-pd-test-kserve-5d9b9b4785 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:09 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-pd-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-pd-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:44 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:48 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [custom-route-timeout-pd-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-d4b5cbd7-ntdnx to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.59/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.59:8000/health": dial tcp 10.133.0.59:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:51 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: custom-route-timeout-test-kserve-d4b5cbd7-ntdnx [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6869bff88sqkk to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.43/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:13 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:13 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:13 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:05 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: custom-route-timeout-test-kserve-router-scheduler-6869bff88sqkk [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set custom-route-timeout-test-kserve-router-scheduler-6869bff8d5 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set custom-route-timeout-test-kserve-d4b5cbd7 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:07 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/custom-route-timeout-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:06:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:37 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [custom-route-timeout-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:51 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/e2e-pvc-model-download-v8gm9 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:51 kserve-ci-e2e-test attachdetach-controller Normal SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-816e24b0-2442-4458-a346-0c49c6eaace7" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:52 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.54/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:52 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:52 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:52 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:45 kserve-ci-e2e-test job-controller Normal SuccessfulCreate Created pod: e2e-pvc-model-download-v8gm9 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:00 kserve-ci-e2e-test job-controller Normal Completed Job completed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:45 kserve-ci-e2e-test persistentvolume-controller Normal WaitForFirstConsumer waiting for first consumer to be created before binding [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:46 kserve-ci-e2e-test persistentvolume-controller Normal ExternalProvisioning Waiting for a volume to be created either by the external provisioner 'ebs.csi.aws.com' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:45 kserve-ci-e2e-test ebs.csi.aws.com_aws-ebs-csi-driver-controller-7b88b554b4-pdrxj_7324cf68-1b94-480e-984f-2f77ea955896 Normal Provisioning External provisioner is provisioning volume for claim "kserve-ci-e2e-test/e2e-pvc-model-storage" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:47 kserve-ci-e2e-test ebs.csi.aws.com_aws-ebs-csi-driver-controller-7b88b554b4-pdrxj_7324cf68-1b94-480e-984f-2f77ea955896 Normal ProvisioningSucceeded Successfully provisioned volume pvc-816e24b0-2442-4458-a346-0c49c6eaace7 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:25 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc55rhwq [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc55rhwq to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:26 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.45/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:26 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:27 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.024s (1.024s including waiting). Image size: 98346788 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:27 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:27 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:48 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:04 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.45:8000/health": dial tcp 10.133.0.45:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:25 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:52 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b74wh6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b74wh6 to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:52 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.32/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:53 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:54 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.024s (1.024s including waiting). Image size: 98346788 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:54 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:54 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:20 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Warning FailedMount MountVolume.SetUp failed for volume "tls-certs" : secret "gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:05 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:52 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:25 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-route-3c960099-kserve-7c7fd556fb9fphn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-7c7fd556fb9fphn to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:26 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.73/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:26 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:26 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:26 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:29 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:29 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:29 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:36 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.73:8000/health": dial tcp 10.133.0.73:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:50 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:06 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.73:8000/health": dial tcp 10.133.0.73:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:25 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-route-3c960099-kserve-7c7fd556fb from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:25 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-3c960099-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-3c960099-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:46 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-route-3c960099] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-route-50bc673d-kserve-7cd67cfb9bhccbp [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-7cd67cfb9bhccbp to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.63/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.63:8001/health": dial tcp 10.133.0.63:8001: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:40 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:40 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-8cl8mwz [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-8cl8mwz to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.64/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.64:8000/health": dial tcp 10.133.0.64:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:40 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-8c6bb85d from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-route-50bc673d-kserve-7cd67cfb9b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:12 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test00d7278d8a22c4e39146a6b0eb840f45-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:26 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-route-50bc673d] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test00d7278d8a22c4e39146a6b0eb840f45-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:30 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bccf9c7bf7rct2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bccf9c7bf7rct2 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.56/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:34 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:34 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:34 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:21 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.56:8000/health": dial tcp 10.133.0.56:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:35 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:51 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.56:8000/health": dial tcp 10.133.0.56:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:30 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bccf9c7bf from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:27 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:30 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:30 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:30 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test21fe6730fe484f3a92b1a16afe1bac8f-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:39 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:31 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-route-87882a8e] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test21fe6730fe484f3a92b1a16afe1bac8f-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.76/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:32 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:33 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:33 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:34 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test statefulset-controller Normal SuccessfulCreate create Pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1 in StatefulSet llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 successful [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.75/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:32 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:33 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:33 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:29:48 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.75:8000/health": dial tcp 10.133.0.75:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:33 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.75:8000/health": dial tcp 10.133.0.75:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test leaderworkerset Normal CreatingRevision Creating revision with key 5667d6c4d8 for a newly created LeaderWorkerSet [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Created leader statefulset llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Replicas are progressing, with 0 groups ready of total 1 groups [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test statefulset-controller Normal SuccessfulCreate create Pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 in StatefulSet llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn successful [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Created worker statefulset for leader pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:29:58 kserve-ci-e2e-test leaderworkerset Normal AllGroupsReady All replicas are ready, with 1 groups ready of total 1 groups [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:21 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-scc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test386bd5808c5c450f11fd8632e1f821fb-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:13 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-route-dc21cb14] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test386bd5808c5c450f11fd8632e1f821fb-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:55 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4l6x9m [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4l6x9m to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:56 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.37/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:56 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:56 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:56 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:56 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Warning ClusterIPNotAllocated Cluster IP [IPv4]: 172.31.102.45 is not allocated; repairing [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:55 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:52 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:55 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:55 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:55 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:55 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:58:08 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:58:08 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-route-e95b1dc1] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6448b6f56-k8nmn to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.52/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:27 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:27 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:27 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:58:33 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.52:8000/health": dial tcp 10.133.0.52:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:08 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:22 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6448b6f56-k8nmn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schew77k4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler-686fbcf988 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schew77k4 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.38/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:08 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:16 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:22 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-6448b6f56 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:58:44 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-with-7ca60146] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:08 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-c8bdfd4c5-b7t9d to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.53/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:58 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:58 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:58 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:13 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.53:8000/health": dial tcp 10.133.0.53:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:25 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:40 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.53:8000/health": dial tcp 10.133.0.53:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-c8bdfd4c5-b7t9d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel4rnh [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler-58787c8b6c from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel4rnh to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.39/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:54 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:54 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:54 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:55 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:55 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:55 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:25 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:37 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-c8bdfd4c5 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:49 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test05addb65ba05195619f26ef266e8fc04-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:59:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:00:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:00:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:00:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:24 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-with-ba4d693a] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:01:24 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test05addb65ba05195619f26ef266e8fc04-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:39 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.61/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:39 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:40 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:40 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:34 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning FailedToRetrieveImagePullSecret Unable to retrieve some image pull secrets (llmisvc-model-pvc-router-manage-2577e7-b09f84b1-dockercfg-r5srj); attempting to pull the image may not succeed. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:49 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning FailedMount MountVolume.SetUp failed for volume "tls-certs" : secret "llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:47 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test statefulset-controller Normal SuccessfulCreate create Pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 in StatefulSet llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 successful [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:35 kserve-ci-e2e-test attachdetach-controller Normal SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-816e24b0-2442-4458-a346-0c49c6eaace7" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:39 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.60/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:39 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:39 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:39 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:10:59 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.60:8000/health": dial tcp 10.133.0.60:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:31 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:46 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.60:8000/health": dial tcp 10.133.0.60:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test leaderworkerset Normal CreatingRevision Creating revision with key 686d468674 for a newly created LeaderWorkerSet [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Created leader statefulset llmisvc-model-pvc-router-manage-2577e794-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Replicas are progressing, with 0 groups ready of total 1 groups [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test statefulset-controller Normal SuccessfulCreate create Pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 in StatefulSet llmisvc-model-pvc-router-manage-2577e794-kserve-mn successful [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Created worker statefulset for leader pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:11:09 kserve-ci-e2e-test leaderworkerset Normal AllGroupsReady All replicas are ready, with 1 groups ready of total 1 groups [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:32 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-2577e794-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-2577e794-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-scc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test37591b20e96e9663d45a730d03070f1e-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:39 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:39 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:19 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-pvc-router-manage-2577e794] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test37591b20e96e9663d45a730d03070f1e-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:13 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dcbpff [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dcbpff to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:16 kserve-ci-e2e-test attachdetach-controller Normal SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-816e24b0-2442-4458-a346-0c49c6eaace7" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:22 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.55/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:12 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.55:8000/health": dial tcp 10.133.0.55:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:27 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:13 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:10 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-59b9d263-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-59b9d263-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-testc9569cf4801efc0ed27b2f25ffaee875-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:30 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:22 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-pvc-router-manage-59b9d263] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testc9569cf4801efc0ed27b2f25ffaee875-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-bbf66b468-wqjwf to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:59 kserve-ci-e2e-test attachdetach-controller Normal SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-816e24b0-2442-4458-a346-0c49c6eaace7" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:05 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.57/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:05 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulling Pulling image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:07 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" in 1.825s (1.825s including waiting). Image size: 166803393 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:07 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:07 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:07 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:07 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:07 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:07:45 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.57:8001/health": dial tcp 10.133.0.57:8001: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:10 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:10 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-pvc-router-manage-e8706282-kserve-bbf66b468-wqjwf [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79t95rn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79t95rn to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:59 kserve-ci-e2e-test attachdetach-controller Normal SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-816e24b0-2442-4458-a346-0c49c6eaace7" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:06 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.58/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:06 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:06 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:06 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:07:45 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.58:8000/health": dial tcp 10.133.0.58:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:10 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:25 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.58:8000/health": dial tcp 10.133.0.58:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79497db4cc from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-pvc-router-manage-e8706282-kserve-bbf66b468 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:53 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-e8706282-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-e8706282-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-testf5d060a5eb39a04e074b78907a1556a6-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:24 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:16 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:07:56 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-pvc-router-manage-e8706282] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:10 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testf5d060a5eb39a04e074b78907a1556a6-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4rn477 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4rn477 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:28 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.58/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:28 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:28 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:28 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:00 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:25 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test80f99418f0f3eb4c80f02e6144a6d4f2-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:00 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-4c1ba212] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test80f99418f0f3eb4c80f02e6144a6d4f2-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fcrnvd [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fcrnvd to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:03 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.35/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:03 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:03 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:03 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:36 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:01 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test76d7f532acb694e4a7bcef75d32cd8a1-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:20 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:35 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-4f8c0978] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:36 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test76d7f532acb694e4a7bcef75d32cd8a1-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddj4z6q [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddj4z6q to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.34/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:36 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58dd from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-testb7025ca4d8a6f8f5b2fd08b5581d2678-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:35 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-a50492e9] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testb7025ca4d8a6f8f5b2fd08b5581d2678-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vcqb5 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.35/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:57 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vcqb5 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-4b931143-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-4b931143-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:55:19 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-router-managed-test-llm-4b931143] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb44dmt9 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb44dmt9 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.32/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:56 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:11 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.132.0.32:8000/health": dial tcp 10.132.0.32:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-5b1e8f15-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-5b1e8f15-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:51 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-router-managed-test-llm-5b1e8f15] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:02 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879qjp5f [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879qjp5f to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.33/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:36 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:02 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:00 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-e45d1f79-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-e45d1f79-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:35 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-router-managed-test-llm-e45d1f79] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:36 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecr5jf to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:03 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.37/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:36 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:45 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schecr5jf [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:02 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler-6db55854f4 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schepj8r4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler-794465cd98 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schepj8r4 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.44/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:35 kserve-ci-e2e-test attachdetach-controller Warning FailedAttachVolume Multi-Attach error for volume "pvc-816e24b0-2442-4458-a346-0c49c6eaace7" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0, llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:31 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:45 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schetglm9 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler-79848c949b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schetglm9 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:28 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.54/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:29 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:29 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:29 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:30 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:30 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:27:30 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:18 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:32 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schev5h4d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:27 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler-6cc69fd994 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schev5h4d to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:28 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.59/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:28 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:28 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:28 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:00 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:09 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche68br6 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.46/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:17 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:17 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:17 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:40 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:49 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche68br6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:15 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler-6f8cf8775b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schehdj9v [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler-7c9f98478b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schehdj9v to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.31/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:36 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:51:45 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-sche6qvh5 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.40/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:16 kserve-ci-e2e-test attachdetach-controller Warning FailedAttachVolume Multi-Attach error for volume "pvc-816e24b0-2442-4458-a346-0c49c6eaace7" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9dcbpff [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:27 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:36 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-sche6qvh5 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:14 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler-6c65b6df8d from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche9fnck to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.41/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:35 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:05:45 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche9fnck [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:03:31 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler-58dc5f6d78 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schebgjvn to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.34/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:36 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:44 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schebgjvn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:02 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler-69655bc8c5 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schetkz8g [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler-77688f7d79 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schetkz8g to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.42/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:57 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:04:59 kserve-ci-e2e-test attachdetach-controller Warning FailedAttachVolume Multi-Attach error for volume "pvc-816e24b0-2442-4458-a346-0c49c6eaace7" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-e8706282-kserve-bbf66b468-wqjwf, llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79t95rn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:10 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:08:19 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep6gtb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler-595f78db49 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep6gtb to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.36/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:37 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:38 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:38 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:56:57 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:10 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schels7bn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:07 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler-6586f4d6cc from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schels7bn to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:08 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.33/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:08 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:08 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:52:08 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:53:56 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:54:09 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-77c65dc7cf-mfh5p to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:14 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.72/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:14 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:14 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:14 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:19 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:34 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.72:8000/health": dial tcp 10.133.0.72:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:25:43 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:25:58 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.72:8000/health": dial tcp 10.133.0.72:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:14 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: lora-multiple-adapters-test-kserve-77c65dc7cf-mfh5p [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:14 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set lora-multiple-adapters-test-kserve-77c65dc7cf from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:13 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy lora-multiple-adapters-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "lora-multiple-adapters-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/lora-multiple-adapters-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:38 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:46 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [lora-multiple-adapters-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/lora-single-adapter-test-kserve-6c7646bb6b-7wx5b to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:15 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.69/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:21:34 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.69:8000/health": dial tcp 10.133.0.69:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:22:42 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:22:57 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.69:8000/health": dial tcp 10.133.0.69:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:14 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: lora-single-adapter-test-kserve-6c7646bb6b-7wx5b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:14 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set lora-single-adapter-test-kserve-6c7646bb6b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:14 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy lora-single-adapter-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/lora-single-adapter-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "lora-single-adapter-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/lora-single-adapter-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/lora-single-adapter-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/lora-single-adapter-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/lora-single-adapter-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:15 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/lora-single-adapter-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/lora-single-adapter-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:21:45 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [lora-single-adapter-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b4979bc47-dll9c to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.46/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:50 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:50 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:50 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:34 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b4979bc47-j2jk4 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.27/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulling Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:50 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" in 4.013s (4.013s including waiting). Image size: 301882321 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:50 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:50 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:54 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:56 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 2.325s (2.325s including waiting). Image size: 98346788 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:56 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:56 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:34 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:49 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.132.0.27:8000/health": dial tcp 10.132.0.27:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:45 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: precise-prefix-cache-test-kserve-b4979bc47-j2jk4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:45 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: precise-prefix-cache-test-kserve-b4979bc47-dll9c [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: precise-prefix-cache-test-kserve-router-scheduler-7b4b884cw59s7 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-7b4b884cw59s7 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.28/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulling Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:50 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" in 3.587s (3.587s including waiting). Image size: 301882321 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:50 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:50 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulling Pulling image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:54 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" in 2.632s (2.632s including waiting). Image size: 179512283 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:01 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:01 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:54 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulling Pulling image "quay.io/opendatahub/llm-d-kv-cache:v0.8.0" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:31 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Successfully pulled image "quay.io/opendatahub/llm-d-kv-cache:v0.8.0" in 37.838s (37.838s including waiting). Image size: 2935159967 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:32 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: tokenizer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:32 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container tokenizer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:01 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:47 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning BackOff Back-off restarting failed container main in pod precise-prefix-cache-test-kserve-router-scheduler-7b4b884cw59s7_kserve-ci-e2e-test(3685774b-11b2-49d5-9395-84ad14bd1823) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:07 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Liveness probe failed: timeout: failed to connect service "10.132.0.28:9003" within 1s: context deadline exceeded [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:34 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container tokenizer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:34 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:46 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: Get "http://10.132.0.28:8082/healthz": dial tcp 10.132.0.28:8082: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:42 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set precise-prefix-cache-test-kserve-router-scheduler-7b4b884c4 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:45 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set precise-prefix-cache-test-kserve-b4979bc47 from 0 to 2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:31 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy precise-prefix-cache-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "precise-prefix-cache-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/precise-prefix-cache-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/precise-prefix-cache-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/precise-prefix-cache-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:46 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/precise-prefix-cache-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:01 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:49:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:33 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [precise-prefix-cache-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:50:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-5c7d67c6bc-dzgk6 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:53 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.78/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:53 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:53 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:53 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:57 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:57 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:57 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:03 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.78:8000/health": dial tcp 10.133.0.78:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:11 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-5c7d67c6bc-nm7qd to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.77/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:54 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:54 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:54 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:40 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.77:8000/health": dial tcp 10.133.0.77:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:53 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:08 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.77:8000/health": dial tcp 10.133.0.77:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: prestop-hook-test-kserve-5c7d67c6bc-nm7qd [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:53 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: prestop-hook-test-kserve-5c7d67c6bc-dzgk6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-6c8475994c-mhqk2 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:51 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.55/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:52 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:52 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:52 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:11 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: prestop-hook-test-kserve-router-scheduler-6c8475994c-mhqk2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set prestop-hook-test-kserve-router-scheduler-6c8475994c from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set prestop-hook-test-kserve-5c7d67c6bc from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/prestop-hook-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/prestop-hook-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/prestop-hook-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/prestop-hook-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-prestop-hook-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/prestop-hook-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/prestop-hook-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/prestop-hook-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:29:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/prestop-hook-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:09 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/prestop-hook-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:29:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/prestop-hook-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:30:51 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [prestop-hook-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:05 kserve-ci-e2e-test LLMInferenceServiceController Warning LLMInferenceServiceNotReady LLMInferenceService [prestop-hook-test] is no longer Ready because of: MainWorkloadReady, WorkloadsReady [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-prestop-hook-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-pmrdl to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:36 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.40/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:36 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:36 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:36 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:48 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: rolling-upgrade-test-kserve-54bc76f55b-pmrdl [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-hjr8m to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:47 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.80/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:47 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:47 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:47 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:03 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning FailedMount MountVolume.SetUp failed for volume "tls-certs" : secret "rolling-upgrade-test-kserve-self-signed-certs" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:49 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:46 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: rolling-upgrade-test-kserve-859df57bcb-hjr8m [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-r9qcv to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:36 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.79/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:36 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:36 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:36 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:48 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:57 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: rolling-upgrade-test-kserve-router-scheduler-745dd558f-r9qcv [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set rolling-upgrade-test-kserve-router-scheduler-745dd558f from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set rolling-upgrade-test-kserve-54bc76f55b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:46 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set rolling-upgrade-test-kserve-859df57bcb from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:29 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy rolling-upgrade-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/rolling-upgrade-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "rolling-upgrade-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/rolling-upgrade-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/rolling-upgrade-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/rolling-upgrade-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-rolling-upgrade-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/rolling-upgrade-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/rolling-upgrade-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/rolling-upgrade-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/rolling-upgrade-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:46 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/rolling-upgrade-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:46 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/rolling-upgrade-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:30 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [rolling-upgrade-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:46 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-rolling-upgrade-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-9csvf to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:17 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.26/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:17 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulling Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:20 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.748s (2.748s including waiting). Image size: 181251556 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:35 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:35 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:35 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:39 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Startup probe failed: Get "http://10.132.0.26:15021/healthz/ready": dial tcp 10.132.0.26:15021: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:39 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-mw42d to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:15 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.39/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:15 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:15 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:15 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:20 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Warning Unhealthy Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-p4vzh to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:56 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.29/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:58:05 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:58:05 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:58:05 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:58:11 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Startup probe failed: Get "http://10.132.0.29:15021/healthz/ready": dial tcp 10.132.0.29:15021: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:14 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:16 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-54c47cdfc7-9csvf [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:37 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-54c47cdfc7-9csvf [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:55 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-54c47cdfc7-p4vzh [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:14 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-54c47cdfc7-p4vzh [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:14 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-54c47cdfc7-mw42d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-8zdl2 to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:47 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.36/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:47 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:47 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:47 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:49 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-prgpl to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:13 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.38/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:13 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:13 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:13 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:22 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-tv79z to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:28 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.31/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:28 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulling Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:30 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.758s (2.758s including waiting). Image size: 181251556 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:31 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:31 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:36 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Warning Unhealthy Startup probe failed: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:58 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-xccwl to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:17 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.25/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:17 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulling Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:20 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.957s (2.957s including waiting). Image size: 181251556 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:20 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:20 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:16 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-799f46c59b-xccwl [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:22 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-799f46c59b-xccwl [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:27 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-799f46c59b-tv79z [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:58 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-799f46c59b-tv79z [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:47 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-799f46c59b-8zdl2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:57:47 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-799f46c59b-8zdl2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:12 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-799f46c59b-prgpl [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:22 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-799f46c59b-prgpl [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:12 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-gateway-1-openshift-default-799f46c59b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:16 kserve-ci-e2e-test service-controller Normal EnsuringLoadBalancer Ensuring load balancer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:14 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-gateway-1-openshift-default-54c47cdfc7 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 17:48:20 kserve-ci-e2e-test service-controller Normal EnsuredLoadBalancer Ensured load balancer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:22 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled down replica set router-gateway-1-openshift-default-799f46c59b from 1 to 0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:14 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled down replica set router-gateway-1-openshift-default-54c47cdfc7 from 1 to 0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:12 kserve-ci-e2e-test gateway_labeler_controller Normal AddedLabel Added label istio.io/rev=openshift-gateway to gateway router-gateway-1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:02:56 kserve-ci-e2e-test GatewayAuthBootstrap Warning ReconcileError Failed to delete gateway PodMonitor: failed to delete PodMonitor: podmonitors.monitoring.coreos.com "router-gateway-1-metrics" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-44z29 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.50/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:23 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-2-openshift-default-54c789bdc6-44z29 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:23 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-2-openshift-default-54c789bdc6-44z29 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-684dc9b4b9-zbq44 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:21 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.51/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:39 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:20 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-2-openshift-default-684dc9b4b9-zbq44 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-gateway-2-openshift-default-54c789bdc6 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test service-controller Normal EnsuringLoadBalancer Ensuring load balancer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:20 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-gateway-2-openshift-default-684dc9b4b9 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:21 kserve-ci-e2e-test service-controller Normal EnsuredLoadBalancer Ensured load balancer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:23 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled down replica set router-gateway-2-openshift-default-54c789bdc6 from 1 to 0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:18 kserve-ci-e2e-test gateway_labeler_controller Normal AddedLabel Added label istio.io/rev=openshift-gateway to gateway router-gateway-2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:01 kserve-ci-e2e-test GatewayAuthBootstrap Warning ReconcileError Failed to delete gateway PodMonitor: failed to delete PodMonitor: podmonitors.monitoring.coreos.com "router-gateway-2-metrics" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-b4685cc89-zfv4b to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.70/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:42 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:42 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:42 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:46 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:46 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:46 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:41 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.70:8001/health": dial tcp 10.133.0.70:8001: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:01 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:01 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-with-refs-pd-test-kserve-b4685cc89-zfv4b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-9dcd56f4c-sqbg5 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.71/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:45 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:45 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:45 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:41 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.71:8000/health": dial tcp 10.133.0.71:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:01 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-with-refs-pd-test-kserve-prefill-9dcd56f4c-sqbg5 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-with-refs-pd-test-kserve-prefill-9dcd56f4c from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-with-refs-pd-test-kserve-router-scheduler-6668cbdbdh7z6f [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6668cbdbdh7z6f to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:42 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.52/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:42 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:42 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:42 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:43 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:43 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:43 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:01 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:15 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-with-refs-pd-test-kserve-router-scheduler-6668cbdbd4 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-with-refs-pd-test-kserve-b4685cc89 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/router-with-refs-pd-test-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-pd-test-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/router-with-refs-pd-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-pd-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:21:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:23:51 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [router-with-refs-pd-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:24:01 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-68cd5c496c-5htr4 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.62/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:36 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:36 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:36 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:11:32 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.62:8000/health": dial tcp 10.133.0.62:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:11:47 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:12:02 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.62:8000/health": dial tcp 10.133.0.62:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-with-refs-test-kserve-68cd5c496c-5htr4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-859f948bc7-vnnjg to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:33 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.45/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:33 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:34 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:34 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:34 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:11:47 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:11:56 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-with-refs-test-kserve-router-scheduler-859f948bc7-vnnjg [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-with-refs-test-kserve-router-scheduler-859f948bc7 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-with-refs-test-kserve-68cd5c496c from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/router-with-refs-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/router-with-refs-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:32 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/router-with-refs-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:09:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:10:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:11:42 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [router-with-refs-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:11:46 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-d7l7h to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:26 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.60/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:26 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:26 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:26 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:29 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-configmap-ref-test-kserve-744bf59fbd-d7l7h [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-configmap-ref-test-kserve-router-scheduler-669ffrln5v [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-669ffrln5v to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:26 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.61/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:26 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:26 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:26 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:29 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:37 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-configmap-ref-test-kserve-router-scheduler-669fffdcd9 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-configmap-ref-test-kserve-744bf59fbd from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:23 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-configmap-ref-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-configmap-ref-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:25 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:51 kserve-ci-e2e-test LLMInferenceServiceController Warning Error Reconciliation failed: failed to reconcile workload: failed to reconcile self-signed certificates secret: failed to reconcile self-signed TLS certificate: failed to get defaults for v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs: context canceled [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:34:51 kserve-ci-e2e-test LLMInferenceServiceController Warning UpdateFailed Failed to update status for LLMInferenceService "scheduler-configmap-ref-test": context canceled [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:27 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-configmap-ref-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-bgg6r to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.62/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:04 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:19 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.132.0.62:8000/health": dial tcp 10.132.0.62:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-custom-template-test-kserve-5bc7bd6864-bgg6r [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-custom-template-test-kserve-router-scheduler-89dmbzl2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dmbzl2 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:51 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.63/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:04 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:13 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-custom-template-test-kserve-router-scheduler-89d9b575b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-custom-template-test-kserve-5bc7bd6864 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:46 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-custom-template-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-custom-template-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-custom-template-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-custom-template-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-custom-template-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-custom-template-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-custom-template-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-custom-template-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-custom-template-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-custom-template-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-custom-template-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-custom-template-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-custom-template-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-custom-template-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:44 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-custom-template-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-custom-template-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-vj8mt to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.41/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:05 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-ha-replicas-test-kserve-5c5cfcf776-vj8mt [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fj5r7j [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-85cd85ft6fh6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fj5r7j to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.82/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:03 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:03 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:03 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:15 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:05 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85ft6fh6 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.64/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:05 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:15 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:15 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fcfc from 0 to 2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-ha-replicas-test-kserve-5c5cfcf776 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:35:55 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-ha-replicas-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-ha-replicas-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:36:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:03 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-ha-replicas-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-mmp8b to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.56/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:48 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:03 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.132.0.56:8000/health": dial tcp 10.132.0.56:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:59 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-inline-config-test-kserve-7c6c58cc96-mmp8b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fc8nwn8 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.57/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:48 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:33:02 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-inline-config-test-kserve-router-scheduler-568fc8nwn8 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-inline-config-test-kserve-router-scheduler-568fc9df99 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:59 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-inline-config-test-kserve-7c6c58cc96 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:59 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-inline-config-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-inline-config-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-inline-config-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:31:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-inline-config-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-inline-config-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-inline-config-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:38 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-inline-config-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:32:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.83/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.65/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.66/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:24 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-pd-migration-test-kserve-router-scheduler-6fc5d87465 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-pd-migration-test-kserve-74c855b64f from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-pd-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-pd-migration-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-v06-pd-migration-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:44 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:58 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-v06-pd-migration-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-859798d76c-b9zvs to ip-10-0-128-180.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.42/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:55 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulling Pulling image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:57 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" in 2.583s (2.583s including waiting). Image size: 166803393 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:57 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:57 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:58 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:58 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:58 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:28 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:28 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:43 kserve-ci-e2e-test kubelet/ip-10-0-128-180.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.42:8001/health": dial tcp 10.134.0.42:8001: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-threshold-migration-test-kserve-859798d76c-b9zvs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc54jckj to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.67/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:55 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:55 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:55 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:28 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-threshold-migration-test-kserve-prefill-5fc54jckj [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-threshold-migration-test-kserve-prefill-5fc599dc49 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-threshold-migration-test-kserve-router-schedxrdl5 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-threshold-migration-test-kserve-router-scheduler-9d7fdfbbd from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-schedxrdl5 to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:55 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.84/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:55 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:55 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:55 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:36 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-threshold-migration-test-kserve-859798d76c from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:46 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-threshold-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-threshold-migration-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:37:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:10 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:09 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:09 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:27 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-v06-threshold-migration-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:38:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-f98bfbff-nln7t to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:28 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.68/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:28 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:32 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:32 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:32 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:38 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.68:8000/health": dial tcp 10.133.0.68:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:51 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:06 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.68:8000/health": dial tcp 10.133.0.68:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-f98bfbff-xltsj to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.65/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:16 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:42 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.65:8000/health": dial tcp 10.133.0.65:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:54 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:11 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: stop-feature-test-kserve-f98bfbff-xltsj [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:28 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: stop-feature-test-kserve-f98bfbff-nln7t [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-546967d655-jljrz to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.47/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:13 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:13 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:13 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:02 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:15 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-546967d655-nrcg7 to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:29 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.49/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:29 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:29 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:29 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:30 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:30 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:30 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:51 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:20:02 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: stop-feature-test-kserve-router-scheduler-546967d655-jljrz [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:28 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: stop-feature-test-kserve-router-scheduler-546967d655-nrcg7 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set stop-feature-test-kserve-router-scheduler-546967d655 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:28 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set stop-feature-test-kserve-router-scheduler-546967d655 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:11 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set stop-feature-test-kserve-f98bfbff from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:28 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set stop-feature-test-kserve-f98bfbff from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:22 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy stop-feature-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "stop-feature-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-stop-feature-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/stop-feature-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/stop-feature-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:12 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/stop-feature-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:36 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:13:36 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:19:50 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [stop-feature-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:54 kserve-ci-e2e-test LLMInferenceServiceController Warning Error Reconciliation failed: failed to reconcile networking: failed to reconcile scheduler: failed to build expected scheduler deployment: failed to attach model artifacts to scheduler deployment: secrets "stop-feature-test-epp-sa-dockercfg-8hpdf" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:15:54 kserve-ci-e2e-test LLMInferenceServiceController Warning LLMInferenceServiceNotReady LLMInferenceService [stop-feature-test] is no longer Ready because of: MainWorkloadReady, PrefillWorkerWorkloadReady, PrefillWorkloadReady, RouterReady, SchedulerWorkloadReady, WorkerWorkloadReady, WorkloadsReady [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:16:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/tls-verification-test-kserve-57c75fd4f5-wsg7t to ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.74/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:25 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:25 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:25 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:12 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.133.0.74:8000/health": dial tcp 10.133.0.74:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:27 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:43 kserve-ci-e2e-test kubelet/ip-10-0-132-31.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.74:8000/health": dial tcp 10.133.0.74:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:21 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: tls-verification-test-kserve-57c75fd4f5-wsg7t [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-59969bd6f5-4v7gg to ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.53/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:86207d1cba5aecf8e7b5bca617ed35061e54e289c7778b383f47cc8d53214f1a" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:23 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:23 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:23 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:27 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:35 kserve-ci-e2e-test kubelet/ip-10-0-142-99.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: tls-verification-test-kserve-router-scheduler-59969bd6f5-4v7gg [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set tls-verification-test-kserve-router-scheduler-59969bd6f5 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:21 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set tls-verification-test-kserve-57c75fd4f5 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:14 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy tls-verification-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/tls-verification-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "tls-verification-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/tls-verification-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/tls-verification-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/tls-verification-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/tls-verification-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-tls-verification-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/tls-verification-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/tls-verification-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/tls-verification-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/tls-verification-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/tls-verification-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:26:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/tls-verification-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:22 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [tls-verification-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-16 18:28:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-tls-verification-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:148 ### Pod scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24 (phase=Running) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:188 #### init-container 'llm-d-routing-sidecar' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:201 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:202 Flag --decoder-use-tls has been deprecated, use --enable-tls=decoder instead [e2e-llm-inference-service] Flag --prefiller-use-tls has been deprecated, use --enable-tls=prefiller instead [e2e-llm-inference-service] {"level":"info","ts":1784227044.1468167,"msg":"Proxy starting","Built on":"","From Git SHA":"unknown"} [e2e-llm-inference-service] {"level":"info","ts":1784227044.14684,"msg":"Proxy configuration","config":"{\"Port\":\"8000\",\"KVConnector\":\"nixlv2\",\"ECConnector\":\"\",\"DataParallelSize\":1,\"MaxIdleConnsPerHost\":1024,\"EnablePrefillerSampling\":false,\"PrefillMaxRetries\":0,\"PrefillRetryBackoff\":200000000,\"UseTLSForPrefiller\":true,\"UseTLSForDecoder\":true,\"UseTLSForEncoder\":false,\"InsecureSkipVerifyForPrefiller\":false,\"InsecureSkipVerifyForEncoder\":false,\"InsecureSkipVerifyForDecoder\":false,\"SecureServing\":true,\"CertPath\":\"/var/run/kserve/tls\",\"MooncakeBootstrapPort\":8998,\"EnableSSRFProtection\":true,\"InferencePoolNamespace\":\"kserve-ci-e2e-test\",\"InferencePoolName\":\"scheduler-v06-pd-migration-test-inference-pool\",\"PoolGroup\":\"inference.networking.x-k8s.io\",\"DecodeChunkSize\":0,\"Tracing\":false,\"MoRIIOWriteMode\":false,\"MoRIIODecodeNotifyPort\":61005,\"MoRIIODecodeHandshakePort\":6301,\"MoRIIODecodePodIP\":\"\",\"MoRIIOParallelDispatch\":false,\"MoRIIOPrefillHandshakePort\":6301,\"MoRIIOPrefillNotifyPort\":61005,\"MoRIIOTPSize\":1,\"MoRIIODPSize\":1,\"MoRIIORemoteHosts\":null,\"MoRIIODPSizeLocal\":0,\"MoRIIODecodeHosts\":null,\"DecoderURL\":\"https://localhost:8001\"}"} [e2e-llm-inference-service] {"level":"info","ts":1784227044.1476219,"logger":"allowlist-validator","msg":"starting SSRF protection allowlist validator","namespace":"kserve-ci-e2e-test","poolName":"scheduler-v06-pd-migration-test-inference-pool","gvr":"inference.networking.x-k8s.io/v1alpha2, Resource=inferencepools"} [e2e-llm-inference-service] {"level":"info","ts":1784227044.1641066,"logger":"allowlist-validator","msg":"InferencePool added","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227044.1724083,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":0,"targets":{}} [e2e-llm-inference-service] {"level":"info","ts":1784227044.1724722,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":0,"targets":{}} [e2e-llm-inference-service] {"level":"info","ts":1784227044.2485368,"logger":"allowlist-validator","msg":"allowlist validator started successfully"} [e2e-llm-inference-service] {"level":"info","ts":1784227044.2496552,"logger":"proxy server on port 8000","msg":"server TLS configured"} [e2e-llm-inference-service] {"level":"info","ts":1784227044.2496734,"logger":"proxy server on port 8000","msg":"starting","addr":"[::]:8000"} [e2e-llm-inference-service] {"level":"info","ts":1784227044.4807703,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":2,"targets":{"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227044.590104,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227045.4883173,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227053.911411,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227053.9154243,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227054.5532818,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227054.5748296,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227054.6431348,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227074.1647098,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227074.174214,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227074.1743078,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:37:59 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227079.3972192,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"78287b6b-8145-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:37:59 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227079.4178932,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"782bf8a2-8145-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:38:03 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227083.4398096,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"7a91a894-8145-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:38:11 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227091.4612203,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"7f599e03-8145-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227104.1657288,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227104.1784666,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227104.178526,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:38:27 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227107.5135086,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"88eacaa6-8145-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227134.1662989,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227134.1789472,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227134.1790028,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:38:59 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227139.5395977,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"9c018162-8145-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227164.1670387,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227164.1749105,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227164.1749606,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227194.1675656,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227194.1799188,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227194.1799793,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:40:03 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227203.595203,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"c22f9cfe-8145-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227224.168039,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227224.176989,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227224.177046,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227254.169047,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227254.178527,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227254.1785877,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227284.1697536,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227284.186061,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227284.1861286,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227314.170206,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227314.17944,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227314.1795022,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:42:03 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227323.6619806,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"09c05845-8146-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227344.170235,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227344.185243,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227344.1853168,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227374.1705449,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227374.1799135,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227374.1799743,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227404.1709862,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227404.1808279,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227404.1808972,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227434.171817,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227434.1819725,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227434.1820345,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:44:03 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227443.7169929,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"514f5949-8146-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:44:08 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227448.8292208,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"545ba601-8146-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:44:08 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227448.8477986,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"545e7d5f-8146-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:44:12 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227452.8676393,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"56c3d2d8-8146-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:44:20 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227460.8871245,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"5b8b8ffa-8146-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227464.1719844,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227464.1810162,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227464.1810794,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:44:36 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227476.9119477,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"6518733b-8146-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227494.1730146,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227494.1827781,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227494.1828387,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:45:08 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227508.935667,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"782ee1ba-8146-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227524.173795,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227524.187409,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227524.1874685,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227554.1743045,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227554.183114,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227554.1831675,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:46:12 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227572.9906704,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"9e5ce049-8146-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227584.1743662,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227584.1849034,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227584.1849632,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227614.1750767,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227614.18401,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227614.1840668,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227644.1755872,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227644.1846626,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227644.1847234,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227674.1759214,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227674.185167,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227674.1852186,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:48:13 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227693.043213,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"e5eb9011-8146-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227704.1760585,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227704.1856086,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227704.1856582,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227734.1765943,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227734.1858284,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227734.1858828,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227764.1775212,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227764.1861644,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227764.1862495,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227794.1778107,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227794.1873844,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227794.1876872,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:50:13 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227813.092598,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"2d799c9a-8147-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:50:18 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227818.193176,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"30841fd8-8147-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:50:18 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227818.2115088,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"3086ef39-8147-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:50:22 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227822.2309806,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"32ec3a6a-8147-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227824.178782,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227824.189182,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227824.189239,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:50:30 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227830.2516794,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"37b412a3-8147-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/16 18:50:46 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227846.2767377,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"41410009-8147-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227854.179509,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227854.1890726,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227854.189128,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:51:18 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227878.3015506,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"5457a95d-8147-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227884.1797633,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227884.1882002,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227884.188282,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227914.1805954,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227914.1911912,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227914.1912527,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:52:22 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784227942.348617,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"7a8471a7-8147-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784227944.181536,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227944.189782,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227944.1898303,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227974.1818075,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784227974.191078,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784227974.1911352,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228004.1819644,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784228004.19088,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228004.190941,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228034.1828141,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784228034.192197,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228034.1922615,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:54:22 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784228062.4038255,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"c2135db9-8147-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1784228064.1837156,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784228064.1925132,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228064.1925735,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228094.1845717,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784228094.193297,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228094.193362,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228124.1854155,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784228124.1948848,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228124.194936,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228154.1863198,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1784228154.1965315,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] {"level":"info","ts":1784228154.1965895,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.65":{},"10.133.0.83":{},"scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt":{}}} [e2e-llm-inference-service] 2026/07/16 18:56:22 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.65 [e2e-llm-inference-service] {"level":"error","ts":1784228182.4553578,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"09a1e5be-8148-11f1-8443-0a580a850053","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/opt/app-root/src/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:2109"} [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:188 #### container 'main' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:201 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:202 I0716 18:37:24.543206 1 config.go:602] "Configuration:" =< [e2e-llm-inference-service] { [e2e-llm-inference-service] "IP": "", [e2e-llm-inference-service] "PodName": "", [e2e-llm-inference-service] "PodNameSpace": "", [e2e-llm-inference-service] "VllmDevMode": false, [e2e-llm-inference-service] "block-size": 16, [e2e-llm-inference-service] "data-parallel-rank": -1, [e2e-llm-inference-service] "data-parallel-size": 1, [e2e-llm-inference-service] "dataset-in-memory": false, [e2e-llm-inference-service] "dataset-path": "", [e2e-llm-inference-service] "dataset-table-name": "llmd", [e2e-llm-inference-service] "dataset-url": "", [e2e-llm-inference-service] "default-embedding-dimensions": 384, [e2e-llm-inference-service] "ec-transfer-config": "", [e2e-llm-inference-service] "enable-kvcache": false, [e2e-llm-inference-service] "enable-prefix-caching": false, [e2e-llm-inference-service] "enable-request-id-headers": false, [e2e-llm-inference-service] "enable-sleep-mode": false, [e2e-llm-inference-service] "enforce-eager": false, [e2e-llm-inference-service] "event-batch-size": 16, [e2e-llm-inference-service] "failure-injection-rate": 0, [e2e-llm-inference-service] "failure-types": null, [e2e-llm-inference-service] "fake-metrics": null, [e2e-llm-inference-service] "fake-metrics-refresh-interval": 100000000, [e2e-llm-inference-service] "global-cache-hit-threshold": 0, [e2e-llm-inference-service] "hash-seed": "", [e2e-llm-inference-service] "inter-token-latency": 0, [e2e-llm-inference-service] "inter-token-latency-std-dev": 0, [e2e-llm-inference-service] "kv-cache-size": 1024, [e2e-llm-inference-service] "kv-cache-transfer-latency": 0, [e2e-llm-inference-service] "kv-cache-transfer-latency-std-dev": 0, [e2e-llm-inference-service] "kv-cache-transfer-time-per-token": 0, [e2e-llm-inference-service] "kv-cache-transfer-time-std-dev": 0, [e2e-llm-inference-service] "latency-calculator": "", [e2e-llm-inference-service] "lora-modules": null, [e2e-llm-inference-service] "max-cpu-loras": 1, [e2e-llm-inference-service] "max-loras": 1, [e2e-llm-inference-service] "max-model-len": 1024, [e2e-llm-inference-service] "max-num-seqs": 5, [e2e-llm-inference-service] "max-tool-call-array-param-length": 5, [e2e-llm-inference-service] "max-tool-call-integer-param": 100, [e2e-llm-inference-service] "max-tool-call-number-param": 100, [e2e-llm-inference-service] "max-waiting-queue-length": 1000, [e2e-llm-inference-service] "min-tool-call-array-param-length": 1, [e2e-llm-inference-service] "min-tool-call-integer-param": 0, [e2e-llm-inference-service] "min-tool-call-number-param": 0, [e2e-llm-inference-service] "mm-encoder-only": false, [e2e-llm-inference-service] "mm-processor-kwargs": "", [e2e-llm-inference-service] "mode": "random", [e2e-llm-inference-service] "model": "facebook/opt-125m", [e2e-llm-inference-service] "object-tool-call-not-required-field-probability": 50, [e2e-llm-inference-service] "port": 8001, [e2e-llm-inference-service] "prefill-overhead": 0, [e2e-llm-inference-service] "prefill-time-per-token": 0, [e2e-llm-inference-service] "prefill-time-std-dev": 0, [e2e-llm-inference-service] "seed": 1784227044542722600, [e2e-llm-inference-service] "self-signed-certs": false, [e2e-llm-inference-service] "served-model-name": [ [e2e-llm-inference-service] "facebook/opt-125m" [e2e-llm-inference-service] ], [e2e-llm-inference-service] "ssl-certfile": "/var/run/kserve/tls/tls.crt", [e2e-llm-inference-service] "ssl-keyfile": "/var/run/kserve/tls/tls.key", [e2e-llm-inference-service] "time-factor-under-load": 1, [e2e-llm-inference-service] "time-to-first-token": 0, [e2e-llm-inference-service] "time-to-first-token-std-dev": 0, [e2e-llm-inference-service] "tool-call-not-required-param-probability": 50, [e2e-llm-inference-service] "uds-socket-path": "/tmp/tokenizer/tokenizer-uds.socket", [e2e-llm-inference-service] "zmq-endpoint": "tcp://127.0.0.1:5557" [e2e-llm-inference-service] } [e2e-llm-inference-service] > [e2e-llm-inference-service] I0716 18:37:24.584655 1 tokenizer.go:104] "Model is not a real HF model, using simulated tokenizer" model="facebook/opt-125m" [e2e-llm-inference-service] I0716 18:37:24.589380 1 context.go:138] "No dataset path or URL provided, using random text for responses" [e2e-llm-inference-service] I0716 18:37:24.589483 1 communication.go:49] "Starting communication layer" [e2e-llm-inference-service] I0716 18:37:24.590053 1 http_server_tls.go:44] "HTTPS server starting with certificate files" cert="/var/run/kserve/tls/tls.crt" key="/var/run/kserve/tls/tls.key" [e2e-llm-inference-service] I0716 18:37:24.590729 1 simulator.go:188] "Start processing routine" [e2e-llm-inference-service] I0716 18:37:24.590816 1 grpc.go:126] "Server starting" protocol="gRPC" port=8001 [e2e-llm-inference-service] I0716 18:37:24.591091 1 http.go:96] "Server starting" protocol="HTTPS" port=8001 [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:148 ### Pod scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt (phase=Running) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:188 #### container 'main' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:201 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:202 I0716 18:37:24.103063 1 config.go:602] "Configuration:" =< [e2e-llm-inference-service] { [e2e-llm-inference-service] "IP": "", [e2e-llm-inference-service] "PodName": "", [e2e-llm-inference-service] "PodNameSpace": "", [e2e-llm-inference-service] "VllmDevMode": false, [e2e-llm-inference-service] "block-size": 16, [e2e-llm-inference-service] "data-parallel-rank": -1, [e2e-llm-inference-service] "data-parallel-size": 1, [e2e-llm-inference-service] "dataset-in-memory": false, [e2e-llm-inference-service] "dataset-path": "", [e2e-llm-inference-service] "dataset-table-name": "llmd", [e2e-llm-inference-service] "dataset-url": "", [e2e-llm-inference-service] "default-embedding-dimensions": 384, [e2e-llm-inference-service] "ec-transfer-config": "", [e2e-llm-inference-service] "enable-kvcache": false, [e2e-llm-inference-service] "enable-prefix-caching": false, [e2e-llm-inference-service] "enable-request-id-headers": false, [e2e-llm-inference-service] "enable-sleep-mode": false, [e2e-llm-inference-service] "enforce-eager": false, [e2e-llm-inference-service] "event-batch-size": 16, [e2e-llm-inference-service] "failure-injection-rate": 0, [e2e-llm-inference-service] "failure-types": null, [e2e-llm-inference-service] "fake-metrics": null, [e2e-llm-inference-service] "fake-metrics-refresh-interval": 100000000, [e2e-llm-inference-service] "global-cache-hit-threshold": 0, [e2e-llm-inference-service] "hash-seed": "", [e2e-llm-inference-service] "inter-token-latency": 0, [e2e-llm-inference-service] "inter-token-latency-std-dev": 0, [e2e-llm-inference-service] "kv-cache-size": 1024, [e2e-llm-inference-service] "kv-cache-transfer-latency": 0, [e2e-llm-inference-service] "kv-cache-transfer-latency-std-dev": 0, [e2e-llm-inference-service] "kv-cache-transfer-time-per-token": 0, [e2e-llm-inference-service] "kv-cache-transfer-time-std-dev": 0, [e2e-llm-inference-service] "latency-calculator": "", [e2e-llm-inference-service] "lora-modules": null, [e2e-llm-inference-service] "max-cpu-loras": 1, [e2e-llm-inference-service] "max-loras": 1, [e2e-llm-inference-service] "max-model-len": 1024, [e2e-llm-inference-service] "max-num-seqs": 5, [e2e-llm-inference-service] "max-tool-call-array-param-length": 5, [e2e-llm-inference-service] "max-tool-call-integer-param": 100, [e2e-llm-inference-service] "max-tool-call-number-param": 100, [e2e-llm-inference-service] "max-waiting-queue-length": 1000, [e2e-llm-inference-service] "min-tool-call-array-param-length": 1, [e2e-llm-inference-service] "min-tool-call-integer-param": 0, [e2e-llm-inference-service] "min-tool-call-number-param": 0, [e2e-llm-inference-service] "mm-encoder-only": false, [e2e-llm-inference-service] "mm-processor-kwargs": "", [e2e-llm-inference-service] "mode": "random", [e2e-llm-inference-service] "model": "facebook/opt-125m", [e2e-llm-inference-service] "object-tool-call-not-required-field-probability": 50, [e2e-llm-inference-service] "port": 8000, [e2e-llm-inference-service] "prefill-overhead": 0, [e2e-llm-inference-service] "prefill-time-per-token": 0, [e2e-llm-inference-service] "prefill-time-std-dev": 0, [e2e-llm-inference-service] "seed": 1784227044102636500, [e2e-llm-inference-service] "self-signed-certs": false, [e2e-llm-inference-service] "served-model-name": [ [e2e-llm-inference-service] "facebook/opt-125m" [e2e-llm-inference-service] ], [e2e-llm-inference-service] "ssl-certfile": "/var/run/kserve/tls/tls.crt", [e2e-llm-inference-service] "ssl-keyfile": "/var/run/kserve/tls/tls.key", [e2e-llm-inference-service] "time-factor-under-load": 1, [e2e-llm-inference-service] "time-to-first-token": 0, [e2e-llm-inference-service] "time-to-first-token-std-dev": 0, [e2e-llm-inference-service] "tool-call-not-required-param-probability": 50, [e2e-llm-inference-service] "uds-socket-path": "/tmp/tokenizer/tokenizer-uds.socket", [e2e-llm-inference-service] "zmq-endpoint": "tcp://127.0.0.1:5557" [e2e-llm-inference-service] } [e2e-llm-inference-service] > [e2e-llm-inference-service] I0716 18:37:24.138357 1 tokenizer.go:104] "Model is not a real HF model, using simulated tokenizer" model="facebook/opt-125m" [e2e-llm-inference-service] I0716 18:37:24.142655 1 context.go:138] "No dataset path or URL provided, using random text for responses" [e2e-llm-inference-service] I0716 18:37:24.142703 1 simulator.go:188] "Start processing routine" [e2e-llm-inference-service] I0716 18:37:24.142710 1 communication.go:49] "Starting communication layer" [e2e-llm-inference-service] I0716 18:37:24.142955 1 http_server_tls.go:44] "HTTPS server starting with certificate files" cert="/var/run/kserve/tls/tls.crt" key="/var/run/kserve/tls/tls.key" [e2e-llm-inference-service] I0716 18:37:24.143039 1 grpc.go:126] "Server starting" protocol="gRPC" port=8000 [e2e-llm-inference-service] I0716 18:37:24.143629 1 http.go:96] "Server starting" protocol="HTTPS" port=8000 [e2e-llm-inference-service] I0716 18:37:59.397140 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:54472\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:37:59.417867 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:54474\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:38:03.439788 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:54488\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:38:11.461185 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:41040\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:38:27.513482 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:37988\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:38:59.539596 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:41630\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:40:03.595136 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:49542\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:42:03.661997 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:57426\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:44:03.717040 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:48644\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:44:08.829259 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:38466\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:44:08.847842 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:38474\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:44:12.867669 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:38478\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:44:20.887185 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:51242\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:44:36.911989 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:38600\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:45:08.935716 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:51316\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:46:12.990721 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:34980\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:48:13.043202 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:46374\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:50:13.092600 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:41956\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:50:18.193157 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:53168\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:50:18.211481 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:53182\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:50:22.230955 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:53190\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:50:30.251646 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:51830\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:50:46.276610 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:49860\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:51:18.301555 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:51158\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:52:22.348592 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:42520\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:54:22.403795 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:48030\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0716 18:56:22.455288 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.65:8000\"<->\"10.133.0.83:54190\": remote error: tls: bad certificate" [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:148 ### Pod scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k (phase=Running) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:188 #### container 'main' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:201 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:202 "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "unknown" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] }, [e2e-llm-inference-service] "InstrumentationLibrary": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/framework/plugins/scheduling", [e2e-llm-inference-service] "Version": "", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "unknown" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] { [e2e-llm-inference-service] "Name": "gateway.request_orchestration", [e2e-llm-inference-service] "SpanContext": { [e2e-llm-inference-service] "TraceID": "cb097b8434fa75ed186b7dab1acafb06", [e2e-llm-inference-service] "SpanID": "04436de8652db8a4", [e2e-llm-inference-service] "TraceFlags": "01", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "Parent": { [e2e-llm-inference-service] "TraceID": "cb097b8434fa75ed186b7dab1acafb06", [e2e-llm-inference-service] "SpanID": "342ffc63df75665f", [e2e-llm-inference-service] "TraceFlags": "01", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "SpanKind": 2, [e2e-llm-inference-service] "StartTime": "2026-07-16T18:50:22.215951043Z", [e2e-llm-inference-service] "EndTime": "2026-07-16T18:50:22.216093767Z", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "target_model", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "facebook/opt-125m" [e2e-llm-inference-service] } [e2e-llm-inference-service] }, [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "request_prio", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "INT64", [e2e-llm-inference-service] "Value": 0 [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ], [e2e-llm-inference-service] "Events": null, [e2e-llm-inference-service] "Links": null, [e2e-llm-inference-service] "Status": { [e2e-llm-inference-service] "Code": "Unset", [e2e-llm-inference-service] "Description": "" [e2e-llm-inference-service] }, [e2e-llm-inference-service] "DroppedAttributes": 0, [e2e-llm-inference-service] "DroppedEvents": 0, [e2e-llm-inference-service] "DroppedLinks": 0, [e2e-llm-inference-service] "ChildSpanCount": 5, [e2e-llm-inference-service] "Resource": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.name", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "llm-d-epp" [e2e-llm-inference-service] } [e2e-llm-inference-service] }, [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.version", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ], [e2e-llm-inference-service] "InstrumentationScope": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/requestcontrol", [e2e-llm-inference-service] "Version": "", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "unknown" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] }, [e2e-llm-inference-service] "InstrumentationLibrary": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/requestcontrol", [e2e-llm-inference-service] "Version": "", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "unknown" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] { [e2e-llm-inference-service] "Name": "gateway.request", [e2e-llm-inference-service] "SpanContext": { [e2e-llm-inference-service] "TraceID": "cb097b8434fa75ed186b7dab1acafb06", [e2e-llm-inference-service] "SpanID": "342ffc63df75665f", [e2e-llm-inference-service] "TraceFlags": "01", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "Parent": { [e2e-llm-inference-service] "TraceID": "00000000000000000000000000000000", [e2e-llm-inference-service] "SpanID": "0000000000000000", [e2e-llm-inference-service] "TraceFlags": "00", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "SpanKind": 2, [e2e-llm-inference-service] "StartTime": "2026-07-16T18:50:22.215903481Z", [e2e-llm-inference-service] "EndTime": "2026-07-16T18:50:22.232170662Z", [e2e-llm-inference-service] "Attributes": null, [e2e-llm-inference-service] "Events": null, [e2e-llm-inference-service] "Links": null, [e2e-llm-inference-service] "Status": { [e2e-llm-inference-service] "Code": "Unset", [e2e-llm-inference-service] "Description": "" [e2e-llm-inference-service] }, [e2e-llm-inference-service] "DroppedAttributes": 0, [e2e-llm-inference-service] "DroppedEvents": 0, [e2e-llm-inference-service] "DroppedLinks": 0, [e2e-llm-inference-service] "ChildSpanCount": 1, [e2e-llm-inference-service] "Resource": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.name", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "llm-d-epp" [e2e-llm-inference-service] } [e2e-llm-inference-service] }, [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.version", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ], [e2e-llm-inference-service] "InstrumentationScope": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/handlers", [e2e-llm-inference-service] "Version": "", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "unknown" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] }, [e2e-llm-inference-service] "InstrumentationLibrary": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/handlers", [e2e-llm-inference-service] "Version": "", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "unknown" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] {"level":"info","ts":1784227830.2361124,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"37e7c0c8-7823-4bf8-8407-d2b0504f49dd"} [e2e-llm-inference-service] {"level":"info","ts":1784227830.2364552,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"37e7c0c8-7823-4bf8-8407-d2b0504f49dd","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784227846.2593355,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"d994300f-a545-429e-ad4e-7b2da97d128d"} [e2e-llm-inference-service] {"level":"info","ts":1784227846.2596576,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"d994300f-a545-429e-ad4e-7b2da97d128d","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784227878.284251,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"4e0a581a-09da-437a-b60b-a6982dfa9057"} [e2e-llm-inference-service] {"level":"info","ts":1784227878.2846172,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"4e0a581a-09da-437a-b60b-a6982dfa9057","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784227942.331625,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"88ee0f56-a1a9-4ba5-ab53-e9cd59c701ec"} [e2e-llm-inference-service] {"level":"info","ts":1784227942.3319304,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"88ee0f56-a1a9-4ba5-ab53-e9cd59c701ec","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784228062.3863416,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"3051f3f7-74e5-43aa-bbce-736beec1e3e0"} [e2e-llm-inference-service] {"level":"info","ts":1784228062.386683,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"3051f3f7-74e5-43aa-bbce-736beec1e3e0","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784228182.4384232,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"c5dfb84d-e30d-46e2-b2b0-d96c5e464891"} [e2e-llm-inference-service] {"level":"info","ts":1784228182.4387558,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"c5dfb84d-e30d-46e2-b2b0-d96c5e464891","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: aa92a329-6577-4d47-b07e-fd54c4a6044a [e2e-llm-inference-service] resourceVersion: '66518' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpoints.kubernetes.io/managed-by: endpoint-controller [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:subsets: {} [e2e-llm-inference-service] subsets: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - ip: 10.132.0.66 [e2e-llm-inference-service] nodeName: ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k [e2e-llm-inference-service] uid: 0683c982-6d4c-4bd1-aa07-1f6be8ea3e55 [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] port: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] port: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] port: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Endpoints [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: fb914a7e-835b-4ab1-8605-eff35a5decb0 [e2e-llm-inference-service] resourceVersion: '66156' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpoints.kubernetes.io/managed-by: endpoint-controller [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:subsets: {} [e2e-llm-inference-service] subsets: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - ip: 10.132.0.65 [e2e-llm-inference-service] nodeName: ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt [e2e-llm-inference-service] uid: 83cb5708-7fed-4211-9449-958f45154f9c [e2e-llm-inference-service] - ip: 10.133.0.83 [e2e-llm-inference-service] nodeName: ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24 [e2e-llm-inference-service] uid: a22a527d-b525-419d-91ba-a9a2f6b39dcc [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: https [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] appProtocol: https [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Endpoints [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24 [e2e-llm-inference-service] generateName: scheduler-v06-pd-migration-test-kserve-74c855b64f- [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: a22a527d-b525-419d-91ba-a9a2f6b39dcc [e2e-llm-inference-service] resourceVersion: '66150' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] pod-template-hash: 74c855b64f [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] k8s.ovn.org/pod-networks: '{"default":{"ip_addresses":["10.133.0.83/23"],"mac_address":"0a:58:0a:85:00:53","gateway_ips":["10.133.0.1"],"routes":[{"dest":"10.132.0.0/14","nextHop":"10.133.0.1"},{"dest":"172.31.0.0/16","nextHop":"10.133.0.1"},{"dest":"169.254.0.5/32","nextHop":"10.133.0.1"},{"dest":"100.64.0.0/16","nextHop":"10.133.0.1"}],"ip_address":"10.133.0.83/23","gateway_ip":"10.133.0.1","role":"primary"}}' [e2e-llm-inference-service] k8s.v1.cni.cncf.io/network-status: "[{\n \"name\": \"ovn-kubernetes\",\n \ [e2e-llm-inference-service] \ \"interface\": \"eth0\",\n \"ips\": [\n \"10.133.0.83\"\n ],\n\ [e2e-llm-inference-service] \ \"mac\": \"0a:58:0a:85:00:53\",\n \"default\": true,\n \"dns\": {}\n\ [e2e-llm-inference-service] }]" [e2e-llm-inference-service] openshift.io/scc: restricted-v2 [e2e-llm-inference-service] seccomp.security.alpha.kubernetes.io/pod: runtime/default [e2e-llm-inference-service] security.openshift.io/validated-scc-subject-type: user [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-74c855b64f [e2e-llm-inference-service] uid: 2856e0c8-e3f1-4213-93dd-013f42b0c163 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: ip-10-0-132-31 [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.ovn.org/pod-networks: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"2856e0c8-e3f1-4213-93dd-013f42b0c163"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8001,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:enableServiceLinks: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"llm-d-routing-sidecar"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAMESPACE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:fieldRef: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: multus-daemon [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.v1.cni.cncf.io/network-status: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kubelet [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] k:{"type":"ContainersReady"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Initialized"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodReadyToStartContainers"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodScheduled"}: [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] k:{"type":"Ready"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:containerStatuses: {} [e2e-llm-inference-service] f:hostIP: {} [e2e-llm-inference-service] f:hostIPs: {} [e2e-llm-inference-service] f:initContainerStatuses: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:phase: {} [e2e-llm-inference-service] f:podIP: {} [e2e-llm-inference-service] f:podIPs: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"ip":"10.133.0.83"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:ip: {} [e2e-llm-inference-service] f:startTime: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: kube-api-access-rbh8h [e2e-llm-inference-service] projected: [e2e-llm-inference-service] sources: [e2e-llm-inference-service] - serviceAccountToken: [e2e-llm-inference-service] expirationSeconds: 3607 [e2e-llm-inference-service] path: token [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: kube-root-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: ca.crt [e2e-llm-inference-service] path: ca.crt [e2e-llm-inference-service] - downwardAPI: [e2e-llm-inference-service] items: [e2e-llm-inference-service] - path: namespace [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: openshift-service-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: service-ca.crt [e2e-llm-inference-service] path: service-ca.crt [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: llm-d-routing-sidecar [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/pd-sidecar [e2e-llm-inference-service] - --port=8000 [e2e-llm-inference-service] - --vllm-port=8001 [e2e-llm-inference-service] - --kv-connector=nixlv2 [e2e-llm-inference-service] - --enable-ssrf-protection=true [e2e-llm-inference-service] - --pool-group=inference.networking.x-k8s.io [e2e-llm-inference-service] - --inference-pool=kserve-ci-e2e-test/scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --secure-proxy=true [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] - --decoder-use-tls=true [e2e-llm-inference-service] - --prefiller-use-tls=true [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: INFERENCE_POOL_NAMESPACE [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] - name: INFERENCE_POOL_NAME [e2e-llm-inference-service] value: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] resources: {} [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: kube-api-access-rbh8h [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 10 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 5 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000690000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8001' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8001 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: kube-api-access-rbh8h [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000690000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] nodeName: ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] seLinuxOptions: [e2e-llm-inference-service] level: s0:c26,c20 [e2e-llm-inference-service] fsGroup: 1000690000 [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-j4fsz [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-kserve-dockercfg-vczbt [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] tolerations: [e2e-llm-inference-service] - key: node.kubernetes.io/not-ready [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/unreachable [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/memory-pressure [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoSchedule [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] enableServiceLinks: true [e2e-llm-inference-service] preemptionPolicy: PreemptLowerPriority [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] phase: Running [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: PodReadyToStartContainers [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] - type: Initialized [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] - type: Ready [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] - type: ContainersReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] - type: PodScheduled [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] hostIP: 10.0.132.31 [e2e-llm-inference-service] hostIPs: [e2e-llm-inference-service] - ip: 10.0.132.31 [e2e-llm-inference-service] podIP: 10.133.0.83 [e2e-llm-inference-service] podIPs: [e2e-llm-inference-service] - ip: 10.133.0.83 [e2e-llm-inference-service] startTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] initContainerStatuses: [e2e-llm-inference-service] - name: llm-d-routing-sidecar [e2e-llm-inference-service] state: [e2e-llm-inference-service] running: [e2e-llm-inference-service] startedAt: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0 [e2e-llm-inference-service] imageID: quay.io/opendatahub/odh-llm-d-router-disagg-sidecar@sha256:4d57f8fe65a63b44f345487af2b831e64ddf7fb01126d4f51cc8cf49b7412d00 [e2e-llm-inference-service] containerID: cri-o://ef8369b9a959790d1ff30ed39cb05bdc6fccc8da125e8157b08a37ace6eeb39f [e2e-llm-inference-service] started: true [e2e-llm-inference-service] resources: {} [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: kube-api-access-rbh8h [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000690000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000690000 [e2e-llm-inference-service] containerStatuses: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] state: [e2e-llm-inference-service] running: [e2e-llm-inference-service] startedAt: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] imageID: ghcr.io/llm-d/llm-d-inference-sim@sha256:bab162bd25e2ed8b15022387cdb223023aeb33be49476af9f0115c0398fb8ff5 [e2e-llm-inference-service] containerID: cri-o://2d6e112fed67c3e7fcab4e4a56237ec2d2f0a4f9370016a7eba78f54debfd345 [e2e-llm-inference-service] started: true [e2e-llm-inference-service] allocatedResources: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: kube-api-access-rbh8h [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000690000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000690000 [e2e-llm-inference-service] qosClass: Burstable [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt [e2e-llm-inference-service] generateName: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847- [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 83cb5708-7fed-4211-9449-958f45154f9c [e2e-llm-inference-service] resourceVersion: '66155' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] pod-template-hash: 78b7f4c847 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] k8s.ovn.org/pod-networks: '{"default":{"ip_addresses":["10.132.0.65/23"],"mac_address":"0a:58:0a:84:00:41","gateway_ips":["10.132.0.1"],"routes":[{"dest":"10.132.0.0/14","nextHop":"10.132.0.1"},{"dest":"172.31.0.0/16","nextHop":"10.132.0.1"},{"dest":"169.254.0.5/32","nextHop":"10.132.0.1"},{"dest":"100.64.0.0/16","nextHop":"10.132.0.1"}],"ip_address":"10.132.0.65/23","gateway_ip":"10.132.0.1","role":"primary"}}' [e2e-llm-inference-service] k8s.v1.cni.cncf.io/network-status: "[{\n \"name\": \"ovn-kubernetes\",\n \ [e2e-llm-inference-service] \ \"interface\": \"eth0\",\n \"ips\": [\n \"10.132.0.65\"\n ],\n\ [e2e-llm-inference-service] \ \"mac\": \"0a:58:0a:84:00:41\",\n \"default\": true,\n \"dns\": {}\n\ [e2e-llm-inference-service] }]" [e2e-llm-inference-service] openshift.io/scc: restricted-v2 [e2e-llm-inference-service] seccomp.security.alpha.kubernetes.io/pod: runtime/default [e2e-llm-inference-service] security.openshift.io/validated-scc-subject-type: user [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847 [e2e-llm-inference-service] uid: f46337e4-a3be-4590-98b7-49167a046559 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: ip-10-0-142-99 [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.ovn.org/pod-networks: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"f46337e4-a3be-4590-98b7-49167a046559"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:enableServiceLinks: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: multus-daemon [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.v1.cni.cncf.io/network-status: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kubelet [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] k:{"type":"ContainersReady"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Initialized"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodReadyToStartContainers"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodScheduled"}: [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] k:{"type":"Ready"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:containerStatuses: {} [e2e-llm-inference-service] f:hostIP: {} [e2e-llm-inference-service] f:hostIPs: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:phase: {} [e2e-llm-inference-service] f:podIP: {} [e2e-llm-inference-service] f:podIPs: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"ip":"10.132.0.65"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:ip: {} [e2e-llm-inference-service] f:startTime: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: kube-api-access-wx27c [e2e-llm-inference-service] projected: [e2e-llm-inference-service] sources: [e2e-llm-inference-service] - serviceAccountToken: [e2e-llm-inference-service] expirationSeconds: 3607 [e2e-llm-inference-service] path: token [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: kube-root-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: ca.crt [e2e-llm-inference-service] path: ca.crt [e2e-llm-inference-service] - downwardAPI: [e2e-llm-inference-service] items: [e2e-llm-inference-service] - path: namespace [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: openshift-service-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: service-ca.crt [e2e-llm-inference-service] path: service-ca.crt [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8000' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: kube-api-access-wx27c [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: File [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000690000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: default [e2e-llm-inference-service] serviceAccount: default [e2e-llm-inference-service] nodeName: ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] seLinuxOptions: [e2e-llm-inference-service] level: s0:c26,c20 [e2e-llm-inference-service] fsGroup: 1000690000 [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-j4fsz [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] tolerations: [e2e-llm-inference-service] - key: node.kubernetes.io/not-ready [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/unreachable [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/memory-pressure [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoSchedule [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] enableServiceLinks: true [e2e-llm-inference-service] preemptionPolicy: PreemptLowerPriority [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] phase: Running [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: PodReadyToStartContainers [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] - type: Initialized [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] - type: Ready [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] - type: ContainersReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] - type: PodScheduled [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] hostIP: 10.0.142.99 [e2e-llm-inference-service] hostIPs: [e2e-llm-inference-service] - ip: 10.0.142.99 [e2e-llm-inference-service] podIP: 10.132.0.65 [e2e-llm-inference-service] podIPs: [e2e-llm-inference-service] - ip: 10.132.0.65 [e2e-llm-inference-service] startTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] containerStatuses: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] state: [e2e-llm-inference-service] running: [e2e-llm-inference-service] startedAt: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] imageID: ghcr.io/llm-d/llm-d-inference-sim@sha256:bab162bd25e2ed8b15022387cdb223023aeb33be49476af9f0115c0398fb8ff5 [e2e-llm-inference-service] containerID: cri-o://6203a48e37284b2f6bd47774af61011d98949ca8d84a0dcf8d8b44e3eca8d030 [e2e-llm-inference-service] started: true [e2e-llm-inference-service] allocatedResources: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: kube-api-access-wx27c [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000690000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000690000 [e2e-llm-inference-service] qosClass: Burstable [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k [e2e-llm-inference-service] generateName: scheduler-v06-pd-migration-test-kserve-router-scheduler-6fc5d87465- [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 0683c982-6d4c-4bd1-aa07-1f6be8ea3e55 [e2e-llm-inference-service] resourceVersion: '66514' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 6fc5d87465 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] app.kubernetes.io/version: 0.9.0 [e2e-llm-inference-service] certificates.kserve.io/expiration-v2: 'true' [e2e-llm-inference-service] k8s.ovn.org/pod-networks: '{"default":{"ip_addresses":["10.132.0.66/23"],"mac_address":"0a:58:0a:84:00:42","gateway_ips":["10.132.0.1"],"routes":[{"dest":"10.132.0.0/14","nextHop":"10.132.0.1"},{"dest":"172.31.0.0/16","nextHop":"10.132.0.1"},{"dest":"169.254.0.5/32","nextHop":"10.132.0.1"},{"dest":"100.64.0.0/16","nextHop":"10.132.0.1"}],"ip_address":"10.132.0.66/23","gateway_ip":"10.132.0.1","role":"primary"}}' [e2e-llm-inference-service] k8s.v1.cni.cncf.io/network-status: "[{\n \"name\": \"ovn-kubernetes\",\n \ [e2e-llm-inference-service] \ \"interface\": \"eth0\",\n \"ips\": [\n \"10.132.0.66\"\n ],\n\ [e2e-llm-inference-service] \ \"mac\": \"0a:58:0a:84:00:42\",\n \"default\": true,\n \"dns\": {}\n\ [e2e-llm-inference-service] }]" [e2e-llm-inference-service] openshift.io/scc: restricted-v2 [e2e-llm-inference-service] seccomp.security.alpha.kubernetes.io/pod: runtime/default [e2e-llm-inference-service] security.openshift.io/validated-scc-subject-type: user [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-6fc5d87465 [e2e-llm-inference-service] uid: 59738792-19d0-4242-8533-e133f2475e7e [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: ip-10-0-142-99 [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.ovn.org/pod-networks: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/version: {} [e2e-llm-inference-service] f:certificates.kserve.io/expiration-v2: {} [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"59738792-19d0-4242-8533-e133f2475e7e"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp/tokenizer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:enableServiceLinks: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tokenizer-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-uds"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: multus-daemon [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.v1.cni.cncf.io/network-status: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kubelet [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] k:{"type":"ContainersReady"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Initialized"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodReadyToStartContainers"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodScheduled"}: [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] k:{"type":"Ready"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:containerStatuses: {} [e2e-llm-inference-service] f:hostIP: {} [e2e-llm-inference-service] f:hostIPs: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:phase: {} [e2e-llm-inference-service] f:podIP: {} [e2e-llm-inference-service] f:podIPs: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"ip":"10.132.0.66"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:ip: {} [e2e-llm-inference-service] f:startTime: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-tmp [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: kube-api-access-zqq7z [e2e-llm-inference-service] projected: [e2e-llm-inference-service] sources: [e2e-llm-inference-service] - serviceAccountToken: [e2e-llm-inference-service] expirationSeconds: 3607 [e2e-llm-inference-service] path: token [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: kube-root-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: ca.crt [e2e-llm-inference-service] path: ca.crt [e2e-llm-inference-service] - downwardAPI: [e2e-llm-inference-service] items: [e2e-llm-inference-service] - path: namespace [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: openshift-service-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: service-ca.crt [e2e-llm-inference-service] path: service-ca.crt [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/epp [e2e-llm-inference-service] - --pool-name [e2e-llm-inference-service] - scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --pool-namespace [e2e-llm-inference-service] - kserve-ci-e2e-test [e2e-llm-inference-service] - --zap-encoder [e2e-llm-inference-service] - json [e2e-llm-inference-service] - --grpc-port [e2e-llm-inference-service] - '9002' [e2e-llm-inference-service] - --grpc-health-port [e2e-llm-inference-service] - '9003' [e2e-llm-inference-service] - --enable-cert-reload=true [e2e-llm-inference-service] - --secure-serving=true [e2e-llm-inference-service] - --model-server-metrics-scheme=https [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --config-text [e2e-llm-inference-service] - "apiVersion: inference.networking.x-k8s.io/v1alpha1\nkind: EndpointPickerConfig\n\ [e2e-llm-inference-service] plugins:\n- type: disagg-headers-handler\n- type: prefill-filter\n- type: decode-filter\n\ [e2e-llm-inference-service] - type: always-disagg-pd-decider\n- parameters:\n deciders:\n prefill:\ [e2e-llm-inference-service] \ always-disagg-pd-decider\n type: disagg-profile-handler\n- type: prefix-cache-scorer\n\ [e2e-llm-inference-service] - type: queue-scorer\n- type: max-score-picker\nschedulingProfiles:\n- name:\ [e2e-llm-inference-service] \ prefill\n plugins:\n - pluginRef: prefill-filter\n - pluginRef: queue-scorer\n\ [e2e-llm-inference-service] \ weight: 2\n - pluginRef: prefix-cache-scorer\n weight: 3\n - pluginRef:\ [e2e-llm-inference-service] \ max-score-picker\n- name: decode\n plugins:\n - pluginRef: decode-filter\n\ [e2e-llm-inference-service] \ - pluginRef: queue-scorer\n weight: 2\n - pluginRef: prefix-cache-scorer\n\ [e2e-llm-inference-service] \ weight: 3\n - pluginRef: max-score-picker\n" [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] containerPort: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] containerPort: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] containerPort: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] containerPort: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] resources: [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] - name: kube-api-access-zqq7z [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: liveness [e2e-llm-inference-service] initialDelaySeconds: 5 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: readiness [e2e-llm-inference-service] initialDelaySeconds: 30 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000690000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] nodeName: ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] seLinuxOptions: [e2e-llm-inference-service] level: s0:c26,c20 [e2e-llm-inference-service] fsGroup: 1000690000 [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-j4fsz [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-epp-sa-dockercfg-nfl29 [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] tolerations: [e2e-llm-inference-service] - key: node.kubernetes.io/not-ready [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/unreachable [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/memory-pressure [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoSchedule [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] enableServiceLinks: true [e2e-llm-inference-service] preemptionPolicy: PreemptLowerPriority [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] phase: Running [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: PodReadyToStartContainers [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] - type: Initialized [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] - type: Ready [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] - type: ContainersReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] - type: PodScheduled [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] hostIP: 10.0.142.99 [e2e-llm-inference-service] hostIPs: [e2e-llm-inference-service] - ip: 10.0.142.99 [e2e-llm-inference-service] podIP: 10.132.0.66 [e2e-llm-inference-service] podIPs: [e2e-llm-inference-service] - ip: 10.132.0.66 [e2e-llm-inference-service] startTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] containerStatuses: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] state: [e2e-llm-inference-service] running: [e2e-llm-inference-service] startedAt: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0 [e2e-llm-inference-service] imageID: quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:7fd1d61cb5505e026eebab0b9653758779c3dee241e37bea92a0bab961a48c50 [e2e-llm-inference-service] containerID: cri-o://2d1bed46a7aed23f2f9c20c3e508fe36cd9855b06f1b12f1c13956686c473790 [e2e-llm-inference-service] started: true [e2e-llm-inference-service] allocatedResources: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] resources: [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] - name: kube-api-access-zqq7z [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000690000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000690000 [e2e-llm-inference-service] qosClass: Burstable [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 1f40fcdd-3014-4704-84cb-f1a37b2c6d8d [e2e-llm-inference-service] resourceVersion: '65902' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] openshift.io/internal-registry-pull-secret-ref: scheduler-v06-pd-migration-test-epp-sa-dockercfg-nfl29 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: openshift.io/image-registry-pull-secrets_service-account-controller [e2e-llm-inference-service] operation: Apply [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:imagePullSecrets: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:openshift.io/internal-registry-pull-secret-ref: {} [e2e-llm-inference-service] f:secrets: [e2e-llm-inference-service] k:{"name":"scheduler-v06-pd-migration-test-epp-sa-dockercfg-nfl29"}: {} [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:secrets: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"default-dockercfg-j4fsz"}: {} [e2e-llm-inference-service] k:{"name":"seaweedfs-s3-creds"}: {} [e2e-llm-inference-service] secrets: [e2e-llm-inference-service] - name: default-dockercfg-j4fsz [e2e-llm-inference-service] - name: seaweedfs-s3-creds [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-epp-sa-dockercfg-nfl29 [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-j4fsz [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-epp-sa-dockercfg-nfl29 [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: ServiceAccount [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: a94e1ab8-5013-4845-a760-6399713b21f3 [e2e-llm-inference-service] resourceVersion: '65860' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] openshift.io/internal-registry-pull-secret-ref: scheduler-v06-pd-migration-test-kserve-dockercfg-vczbt [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: openshift.io/image-registry-pull-secrets_service-account-controller [e2e-llm-inference-service] operation: Apply [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:imagePullSecrets: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:openshift.io/internal-registry-pull-secret-ref: {} [e2e-llm-inference-service] f:secrets: [e2e-llm-inference-service] k:{"name":"scheduler-v06-pd-migration-test-kserve-dockercfg-vczbt"}: {} [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:secrets: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"default-dockercfg-j4fsz"}: {} [e2e-llm-inference-service] k:{"name":"seaweedfs-s3-creds"}: {} [e2e-llm-inference-service] secrets: [e2e-llm-inference-service] - name: default-dockercfg-j4fsz [e2e-llm-inference-service] - name: seaweedfs-s3-creds [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-kserve-dockercfg-vczbt [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-j4fsz [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-kserve-dockercfg-vczbt [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: ServiceAccount [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 43d6863e-8125-4b02-b2de-9ae179452189 [e2e-llm-inference-service] resourceVersion: '65923' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:internalTrafficPolicy: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"port":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] k:{"port":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] k:{"port":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] k:{"port":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:sessionAffinity: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] spec: [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9002 [e2e-llm-inference-service] targetPort: grpc [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] targetPort: grpc-health [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9090 [e2e-llm-inference-service] targetPort: metrics [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 5557 [e2e-llm-inference-service] targetPort: zmq [e2e-llm-inference-service] selector: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] clusterIP: 172.31.144.199 [e2e-llm-inference-service] clusterIPs: [e2e-llm-inference-service] - 172.31.144.199 [e2e-llm-inference-service] type: ClusterIP [e2e-llm-inference-service] sessionAffinity: None [e2e-llm-inference-service] ipFamilies: [e2e-llm-inference-service] - IPv4 [e2e-llm-inference-service] ipFamilyPolicy: SingleStack [e2e-llm-inference-service] internalTrafficPolicy: Cluster [e2e-llm-inference-service] status: [e2e-llm-inference-service] loadBalancer: {} [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 929ae321-5b86-4000-bd4b-bf9eab0e516e [e2e-llm-inference-service] resourceVersion: '65886' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:internalTrafficPolicy: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"port":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:appProtocol: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:sessionAffinity: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] spec: [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: https [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] appProtocol: https [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] targetPort: 8000 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] clusterIP: 172.31.251.241 [e2e-llm-inference-service] clusterIPs: [e2e-llm-inference-service] - 172.31.251.241 [e2e-llm-inference-service] type: ClusterIP [e2e-llm-inference-service] sessionAffinity: None [e2e-llm-inference-service] ipFamilies: [e2e-llm-inference-service] - IPv4 [e2e-llm-inference-service] ipFamilyPolicy: SingleStack [e2e-llm-inference-service] internalTrafficPolicy: Cluster [e2e-llm-inference-service] status: [e2e-llm-inference-service] loadBalancer: {} [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 95e98066-946c-4da6-8553-3af4b70038f2 [e2e-llm-inference-service] resourceVersion: '66154' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:progressDeadlineSeconds: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:revisionHistoryLimit: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:strategy: [e2e-llm-inference-service] f:rollingUpdate: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:maxSurge: {} [e2e-llm-inference-service] f:maxUnavailable: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8001,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"llm-d-routing-sidecar"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAMESPACE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:fieldRef: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"type":"Available"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Progressing"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:updatedReplicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: llm-d-routing-sidecar [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/pd-sidecar [e2e-llm-inference-service] - --port=8000 [e2e-llm-inference-service] - --vllm-port=8001 [e2e-llm-inference-service] - --kv-connector=nixlv2 [e2e-llm-inference-service] - --enable-ssrf-protection=true [e2e-llm-inference-service] - --pool-group=inference.networking.x-k8s.io [e2e-llm-inference-service] - --inference-pool=kserve-ci-e2e-test/scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --secure-proxy=true [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] - --decoder-use-tls=true [e2e-llm-inference-service] - --prefiller-use-tls=true [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: INFERENCE_POOL_NAMESPACE [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] - name: INFERENCE_POOL_NAME [e2e-llm-inference-service] value: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] resources: {} [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 10 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 5 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8001' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8001 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] strategy: [e2e-llm-inference-service] type: RollingUpdate [e2e-llm-inference-service] rollingUpdate: [e2e-llm-inference-service] maxUnavailable: 25% [e2e-llm-inference-service] maxSurge: 25% [e2e-llm-inference-service] revisionHistoryLimit: 10 [e2e-llm-inference-service] progressDeadlineSeconds: 600 [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] updatedReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: Available [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] reason: MinimumReplicasAvailable [e2e-llm-inference-service] message: Deployment has minimum availability. [e2e-llm-inference-service] - type: Progressing [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] reason: NewReplicaSetAvailable [e2e-llm-inference-service] message: ReplicaSet "scheduler-v06-pd-migration-test-kserve-74c855b64f" has successfully [e2e-llm-inference-service] progressed. [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 73bb3e8a-5c5e-47b2-9140-4bd658439090 [e2e-llm-inference-service] resourceVersion: '66159' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:progressDeadlineSeconds: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:revisionHistoryLimit: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:strategy: [e2e-llm-inference-service] f:rollingUpdate: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:maxSurge: {} [e2e-llm-inference-service] f:maxUnavailable: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"type":"Available"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Progressing"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:updatedReplicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8000' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: File [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] strategy: [e2e-llm-inference-service] type: RollingUpdate [e2e-llm-inference-service] rollingUpdate: [e2e-llm-inference-service] maxUnavailable: 25% [e2e-llm-inference-service] maxSurge: 25% [e2e-llm-inference-service] revisionHistoryLimit: 10 [e2e-llm-inference-service] progressDeadlineSeconds: 600 [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] updatedReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: Available [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] reason: MinimumReplicasAvailable [e2e-llm-inference-service] message: Deployment has minimum availability. [e2e-llm-inference-service] - type: Progressing [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] reason: NewReplicaSetAvailable [e2e-llm-inference-service] message: ReplicaSet "scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847" [e2e-llm-inference-service] has successfully progressed. [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 7d0e5d05-30d0-4ac3-8c43-2666c90c263e [e2e-llm-inference-service] resourceVersion: '66522' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:progressDeadlineSeconds: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:revisionHistoryLimit: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:strategy: [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/version: {} [e2e-llm-inference-service] f:certificates.kserve.io/expiration-v2: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp/tokenizer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tokenizer-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-uds"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"type":"Available"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Progressing"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:updatedReplicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] app.kubernetes.io/version: 0.9.0 [e2e-llm-inference-service] certificates.kserve.io/expiration-v2: 'true' [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-tmp [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/epp [e2e-llm-inference-service] - --pool-name [e2e-llm-inference-service] - scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --pool-namespace [e2e-llm-inference-service] - kserve-ci-e2e-test [e2e-llm-inference-service] - --zap-encoder [e2e-llm-inference-service] - json [e2e-llm-inference-service] - --grpc-port [e2e-llm-inference-service] - '9002' [e2e-llm-inference-service] - --grpc-health-port [e2e-llm-inference-service] - '9003' [e2e-llm-inference-service] - --enable-cert-reload=true [e2e-llm-inference-service] - --secure-serving=true [e2e-llm-inference-service] - --model-server-metrics-scheme=https [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --config-text [e2e-llm-inference-service] - "apiVersion: inference.networking.x-k8s.io/v1alpha1\nkind: EndpointPickerConfig\n\ [e2e-llm-inference-service] plugins:\n- type: disagg-headers-handler\n- type: prefill-filter\n- type:\ [e2e-llm-inference-service] \ decode-filter\n- type: always-disagg-pd-decider\n- parameters:\n deciders:\n\ [e2e-llm-inference-service] \ prefill: always-disagg-pd-decider\n type: disagg-profile-handler\n\ [e2e-llm-inference-service] - type: prefix-cache-scorer\n- type: queue-scorer\n- type: max-score-picker\n\ [e2e-llm-inference-service] schedulingProfiles:\n- name: prefill\n plugins:\n - pluginRef: prefill-filter\n\ [e2e-llm-inference-service] \ - pluginRef: queue-scorer\n weight: 2\n - pluginRef: prefix-cache-scorer\n\ [e2e-llm-inference-service] \ weight: 3\n - pluginRef: max-score-picker\n- name: decode\n plugins:\n\ [e2e-llm-inference-service] \ - pluginRef: decode-filter\n - pluginRef: queue-scorer\n weight:\ [e2e-llm-inference-service] \ 2\n - pluginRef: prefix-cache-scorer\n weight: 3\n - pluginRef: max-score-picker\n" [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] containerPort: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] containerPort: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] containerPort: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] containerPort: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] resources: [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: liveness [e2e-llm-inference-service] initialDelaySeconds: 5 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: readiness [e2e-llm-inference-service] initialDelaySeconds: 30 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] strategy: [e2e-llm-inference-service] type: Recreate [e2e-llm-inference-service] revisionHistoryLimit: 10 [e2e-llm-inference-service] progressDeadlineSeconds: 600 [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] updatedReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: Available [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] reason: MinimumReplicasAvailable [e2e-llm-inference-service] message: Deployment has minimum availability. [e2e-llm-inference-service] - type: Progressing [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] reason: NewReplicaSetAvailable [e2e-llm-inference-service] message: ReplicaSet "scheduler-v06-pd-migration-test-kserve-router-scheduler-6fc5d87465" [e2e-llm-inference-service] has successfully progressed. [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-74c855b64f [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 2856e0c8-e3f1-4213-93dd-013f42b0c163 [e2e-llm-inference-service] resourceVersion: '66153' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] pod-template-hash: 74c855b64f [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/desired-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/max-replicas: '2' [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] uid: 95e98066-946c-4da6-8553-3af4b70038f2 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/desired-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/max-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"95e98066-946c-4da6-8553-3af4b70038f2"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8001,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"llm-d-routing-sidecar"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAMESPACE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:fieldRef: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:fullyLabeledReplicas: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] pod-template-hash: 74c855b64f [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] pod-template-hash: 74c855b64f [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: llm-d-routing-sidecar [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/pd-sidecar [e2e-llm-inference-service] - --port=8000 [e2e-llm-inference-service] - --vllm-port=8001 [e2e-llm-inference-service] - --kv-connector=nixlv2 [e2e-llm-inference-service] - --enable-ssrf-protection=true [e2e-llm-inference-service] - --pool-group=inference.networking.x-k8s.io [e2e-llm-inference-service] - --inference-pool=kserve-ci-e2e-test/scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --secure-proxy=true [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] - --decoder-use-tls=true [e2e-llm-inference-service] - --prefiller-use-tls=true [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: INFERENCE_POOL_NAMESPACE [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] - name: INFERENCE_POOL_NAME [e2e-llm-inference-service] value: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] resources: {} [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 10 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 5 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8001' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8001 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] status: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] fullyLabeledReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847 [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: f46337e4-a3be-4590-98b7-49167a046559 [e2e-llm-inference-service] resourceVersion: '66158' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] pod-template-hash: 78b7f4c847 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/desired-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/max-replicas: '2' [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill [e2e-llm-inference-service] uid: 73bb3e8a-5c5e-47b2-9140-4bd658439090 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/desired-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/max-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"73bb3e8a-5c5e-47b2-9140-4bd658439090"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:fullyLabeledReplicas: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] pod-template-hash: 78b7f4c847 [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] pod-template-hash: 78b7f4c847 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8000' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: File [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] status: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] fullyLabeledReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-6fc5d87465 [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 59738792-19d0-4242-8533-e133f2475e7e [e2e-llm-inference-service] resourceVersion: '66519' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 6fc5d87465 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/desired-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/max-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler [e2e-llm-inference-service] uid: 7d0e5d05-30d0-4ac3-8c43-2666c90c263e [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/desired-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/max-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"7d0e5d05-30d0-4ac3-8c43-2666c90c263e"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/version: {} [e2e-llm-inference-service] f:certificates.kserve.io/expiration-v2: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp/tokenizer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tokenizer-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-uds"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:fullyLabeledReplicas: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 6fc5d87465 [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 6fc5d87465 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] app.kubernetes.io/version: 0.9.0 [e2e-llm-inference-service] certificates.kserve.io/expiration-v2: 'true' [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-tmp [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/epp [e2e-llm-inference-service] - --pool-name [e2e-llm-inference-service] - scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --pool-namespace [e2e-llm-inference-service] - kserve-ci-e2e-test [e2e-llm-inference-service] - --zap-encoder [e2e-llm-inference-service] - json [e2e-llm-inference-service] - --grpc-port [e2e-llm-inference-service] - '9002' [e2e-llm-inference-service] - --grpc-health-port [e2e-llm-inference-service] - '9003' [e2e-llm-inference-service] - --enable-cert-reload=true [e2e-llm-inference-service] - --secure-serving=true [e2e-llm-inference-service] - --model-server-metrics-scheme=https [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --config-text [e2e-llm-inference-service] - "apiVersion: inference.networking.x-k8s.io/v1alpha1\nkind: EndpointPickerConfig\n\ [e2e-llm-inference-service] plugins:\n- type: disagg-headers-handler\n- type: prefill-filter\n- type:\ [e2e-llm-inference-service] \ decode-filter\n- type: always-disagg-pd-decider\n- parameters:\n deciders:\n\ [e2e-llm-inference-service] \ prefill: always-disagg-pd-decider\n type: disagg-profile-handler\n\ [e2e-llm-inference-service] - type: prefix-cache-scorer\n- type: queue-scorer\n- type: max-score-picker\n\ [e2e-llm-inference-service] schedulingProfiles:\n- name: prefill\n plugins:\n - pluginRef: prefill-filter\n\ [e2e-llm-inference-service] \ - pluginRef: queue-scorer\n weight: 2\n - pluginRef: prefix-cache-scorer\n\ [e2e-llm-inference-service] \ weight: 3\n - pluginRef: max-score-picker\n- name: decode\n plugins:\n\ [e2e-llm-inference-service] \ - pluginRef: decode-filter\n - pluginRef: queue-scorer\n weight:\ [e2e-llm-inference-service] \ 2\n - pluginRef: prefix-cache-scorer\n weight: 3\n - pluginRef: max-score-picker\n" [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] containerPort: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] containerPort: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] containerPort: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] containerPort: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] resources: [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: liveness [e2e-llm-inference-service] initialDelaySeconds: 5 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: readiness [e2e-llm-inference-service] initialDelaySeconds: 30 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] status: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] fullyLabeledReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-rb [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 627f6a03-3a2b-4495-a514-e1ec6247c373 [e2e-llm-inference-service] resourceVersion: '65917' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:roleRef: {} [e2e-llm-inference-service] f:subjects: {} [e2e-llm-inference-service] subjects: [e2e-llm-inference-service] - kind: ServiceAccount [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] roleRef: [e2e-llm-inference-service] apiGroup: rbac.authorization.k8s.io [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-role [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] kind: RoleBinding [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-rb [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: b7865410-a014-45fc-b6d0-d0d26bff7e1e [e2e-llm-inference-service] resourceVersion: '65868' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:roleRef: {} [e2e-llm-inference-service] f:subjects: {} [e2e-llm-inference-service] subjects: [e2e-llm-inference-service] - kind: ServiceAccount [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] roleRef: [e2e-llm-inference-service] apiGroup: rbac.authorization.k8s.io [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-role [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] kind: RoleBinding [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-role [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 5ccf264b-8c30-469f-8826-cd07aacd0d80 [e2e-llm-inference-service] resourceVersion: '65913' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:rules: {} [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - '' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - pods [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.k8s.io [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencepools [e2e-llm-inference-service] - inferenceobjectives [e2e-llm-inference-service] - inferencemodels [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencemodelrewrites [e2e-llm-inference-service] - inferencepoolimports [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - discovery.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - endpointslices [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] - create [e2e-llm-inference-service] - update [e2e-llm-inference-service] - patch [e2e-llm-inference-service] - delete [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - coordination.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - leases [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-role [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 1e3f6a28-8d6d-49ef-8b4f-14a5b4f0b2a1 [e2e-llm-inference-service] resourceVersion: '65864' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:rules: {} [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - '' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - pods [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] - inference.networking.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencepools [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service-vcfhc [e2e-llm-inference-service] generateName: scheduler-v06-pd-migration-test-epp-service- [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: f33de315-27b8-4234-ae72-959a65a23a05 [e2e-llm-inference-service] resourceVersion: '66517' [e2e-llm-inference-service] generation: 3 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io [e2e-llm-inference-service] kubernetes.io/service-name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] uid: 43d6863e-8125-4b02-b2de-9ae179452189 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:55Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:addressType: {} [e2e-llm-inference-service] f:endpoints: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpointslice.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:kubernetes.io/service-name: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"43d6863e-8125-4b02-b2de-9ae179452189"}: {} [e2e-llm-inference-service] f:ports: {} [e2e-llm-inference-service] addressType: IPv4 [e2e-llm-inference-service] endpoints: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - 10.132.0.66 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] serving: true [e2e-llm-inference-service] terminating: false [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k [e2e-llm-inference-service] uid: 0683c982-6d4c-4bd1-aa07-1f6be8ea3e55 [e2e-llm-inference-service] nodeName: ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] zone: us-east-1a [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9002 [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9090 [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 5557 [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] kind: EndpointSlice [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc-qnh9c [e2e-llm-inference-service] generateName: scheduler-v06-pd-migration-test-kserve-workload-svc- [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 10f9cdb6-f4fb-47f0-90b9-2071d096b124 [e2e-llm-inference-service] resourceVersion: '66157' [e2e-llm-inference-service] generation: 5 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io [e2e-llm-inference-service] kubernetes.io/service-name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] uid: 929ae321-5b86-4000-bd4b-bf9eab0e516e [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:34Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:addressType: {} [e2e-llm-inference-service] f:endpoints: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpointslice.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:kubernetes.io/service-name: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"929ae321-5b86-4000-bd4b-bf9eab0e516e"}: {} [e2e-llm-inference-service] f:ports: {} [e2e-llm-inference-service] addressType: IPv4 [e2e-llm-inference-service] endpoints: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - 10.133.0.83 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] serving: true [e2e-llm-inference-service] terminating: false [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24 [e2e-llm-inference-service] uid: a22a527d-b525-419d-91ba-a9a2f6b39dcc [e2e-llm-inference-service] nodeName: ip-10-0-132-31.ec2.internal [e2e-llm-inference-service] zone: us-east-1a [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - 10.132.0.65 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] serving: true [e2e-llm-inference-service] terminating: false [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt [e2e-llm-inference-service] uid: 83cb5708-7fed-4211-9449-958f45154f9c [e2e-llm-inference-service] nodeName: ip-10-0-142-99.ec2.internal [e2e-llm-inference-service] zone: us-east-1a [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: https [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] appProtocol: https [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] kind: EndpointSlice [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-rb [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 627f6a03-3a2b-4495-a514-e1ec6247c373 [e2e-llm-inference-service] resourceVersion: '65917' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:roleRef: {} [e2e-llm-inference-service] f:subjects: {} [e2e-llm-inference-service] userNames: [e2e-llm-inference-service] - system:serviceaccount:kserve-ci-e2e-test:scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] groupNames: null [e2e-llm-inference-service] subjects: [e2e-llm-inference-service] - kind: ServiceAccount [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] roleRef: [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-role [e2e-llm-inference-service] apiVersion: authorization.openshift.io/v1 [e2e-llm-inference-service] kind: RoleBinding [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-rb [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: b7865410-a014-45fc-b6d0-d0d26bff7e1e [e2e-llm-inference-service] resourceVersion: '65868' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:roleRef: {} [e2e-llm-inference-service] f:subjects: {} [e2e-llm-inference-service] userNames: [e2e-llm-inference-service] - system:serviceaccount:kserve-ci-e2e-test:scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] groupNames: null [e2e-llm-inference-service] subjects: [e2e-llm-inference-service] - kind: ServiceAccount [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] roleRef: [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-role [e2e-llm-inference-service] apiVersion: authorization.openshift.io/v1 [e2e-llm-inference-service] kind: RoleBinding [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-role [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 5ccf264b-8c30-469f-8826-cd07aacd0d80 [e2e-llm-inference-service] resourceVersion: '65913' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:rules: {} [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - '' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - pods [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.k8s.io [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencemodels [e2e-llm-inference-service] - inferenceobjectives [e2e-llm-inference-service] - inferencepools [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencemodelrewrites [e2e-llm-inference-service] - inferencepoolimports [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - discovery.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - endpointslices [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - create [e2e-llm-inference-service] - delete [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - patch [e2e-llm-inference-service] - update [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - coordination.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - leases [e2e-llm-inference-service] apiVersion: authorization.openshift.io/v1 [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-role [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 1e3f6a28-8d6d-49ef-8b4f-14a5b4f0b2a1 [e2e-llm-inference-service] resourceVersion: '65864' [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:rules: {} [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - '' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - pods [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] - inference.networking.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencepools [e2e-llm-inference-service] apiVersion: authorization.openshift.io/v1 [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] kind: HTTPRoute [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] serving.kserve.io/inference-pool-migrated: v1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] generation: 2 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:serving.kserve.io/inference-pool-migrated: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1beta1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] manager: pilot-discovery [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:parents: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-16T18:37:45Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-route [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '66281' [e2e-llm-inference-service] uid: bcd63eeb-c078-4eb6-82f3-52009739566f [e2e-llm-inference-service] spec: [e2e-llm-inference-service] parentRefs: [e2e-llm-inference-service] - group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/chat/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/chat/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/responses [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/responses [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/messages [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/messages [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: / [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: / [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] status: [e2e-llm-inference-service] parents: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] message: Route was valid [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] message: All references resolved [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: ResolvedRefs [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: ResolvedRefs [e2e-llm-inference-service] controllerName: openshift.io/gateway-controller/v1 [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] message: Object affected by AuthPolicy [kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route-authn [e2e-llm-inference-service] openshift-ingress/openshift-ai-inference-authn] [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: kuadrant.io/AuthPolicyAffected [e2e-llm-inference-service] controllerName: kuadrant.io/policy-controller [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: gateway.networking.k8s.io/v1beta1 [e2e-llm-inference-service] kind: HTTPRoute [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] serving.kserve.io/inference-pool-migrated: v1 [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] generation: 2 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:serving.kserve.io/inference-pool-migrated: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1beta1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] manager: pilot-discovery [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:parents: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-16T18:37:45Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-route [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '66281' [e2e-llm-inference-service] uid: bcd63eeb-c078-4eb6-82f3-52009739566f [e2e-llm-inference-service] spec: [e2e-llm-inference-service] parentRefs: [e2e-llm-inference-service] - group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/chat/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/chat/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/responses [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/responses [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/messages [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/messages [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: / [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: / [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] status: [e2e-llm-inference-service] parents: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] message: Route was valid [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] message: All references resolved [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: ResolvedRefs [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: ResolvedRefs [e2e-llm-inference-service] controllerName: openshift.io/gateway-controller/v1 [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] message: Object affected by AuthPolicy [kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route-authn [e2e-llm-inference-service] openshift-ingress/openshift-ai-inference-authn] [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: kuadrant.io/AuthPolicyAffected [e2e-llm-inference-service] controllerName: kuadrant.io/policy-controller [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: inference.networking.k8s.io/v1 [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: inference.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:appProtocol: {} [e2e-llm-inference-service] f:endpointPickerRef: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureMode: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:number: {} [e2e-llm-inference-service] f:selector: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:matchLabels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:targetPorts: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] - apiVersion: inference.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:parents: {} [e2e-llm-inference-service] manager: pilot-discovery [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '66265' [e2e-llm-inference-service] uid: 82795817-0ce2-4a61-bd12-21ca2bc02eab [e2e-llm-inference-service] spec: [e2e-llm-inference-service] appProtocol: http [e2e-llm-inference-service] endpointPickerRef: [e2e-llm-inference-service] failureMode: FailOpen [e2e-llm-inference-service] group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] port: [e2e-llm-inference-service] number: 9002 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] targetPorts: [e2e-llm-inference-service] - number: 8000 [e2e-llm-inference-service] status: [e2e-llm-inference-service] parents: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] message: Referenced by an HTTPRoute accepted by the parentRef Gateway [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] message: Referenced ExtensionRef resolved successfully [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] reason: ResolvedRefs [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: ResolvedRefs [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: networking.istio.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: kuadrant.io/v1 [e2e-llm-inference-service] kind: AuthPolicy [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-policies [e2e-llm-inference-service] app.kubernetes.io/managed-by: odh-model-controller [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: kuadrant.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:rules: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:authentication: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:public: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:anonymous: {} [e2e-llm-inference-service] f:credentials: {} [e2e-llm-inference-service] f:metrics: {} [e2e-llm-inference-service] f:overrides: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:fairness: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:objective: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:priority: {} [e2e-llm-inference-service] f:response: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:success: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:headers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:x-gateway-inference-fairness-id: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:metrics: {} [e2e-llm-inference-service] f:plain: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:expression: {} [e2e-llm-inference-service] f:priority: {} [e2e-llm-inference-service] f:x-gateway-inference-objective: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:metrics: {} [e2e-llm-inference-service] f:plain: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:expression: {} [e2e-llm-inference-service] f:priority: {} [e2e-llm-inference-service] f:targetRef: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] - apiVersion: kuadrant.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"type":"Accepted"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Enforced"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-16T18:37:25Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-route-authn [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '66042' [e2e-llm-inference-service] uid: 2f7fc3af-56d8-4a1a-a81d-f8932c53d2e3 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] rules: [e2e-llm-inference-service] authentication: [e2e-llm-inference-service] public: [e2e-llm-inference-service] anonymous: {} [e2e-llm-inference-service] credentials: {} [e2e-llm-inference-service] metrics: false [e2e-llm-inference-service] overrides: [e2e-llm-inference-service] fairness: [e2e-llm-inference-service] value: unauthenticated [e2e-llm-inference-service] objective: [e2e-llm-inference-service] value: unauthenticated [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] response: [e2e-llm-inference-service] success: [e2e-llm-inference-service] headers: [e2e-llm-inference-service] x-gateway-inference-fairness-id: [e2e-llm-inference-service] metrics: false [e2e-llm-inference-service] plain: [e2e-llm-inference-service] expression: auth.identity.fairness [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] x-gateway-inference-objective: [e2e-llm-inference-service] metrics: false [e2e-llm-inference-service] plain: [e2e-llm-inference-service] expression: auth.identity.objective [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: HTTPRoute [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-route [e2e-llm-inference-service] status: [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] message: AuthPolicy has been accepted [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-16T18:37:25Z' [e2e-llm-inference-service] message: AuthPolicy has been successfully enforced [e2e-llm-inference-service] reason: Enforced [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Enforced [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-scheduler [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '65985' [e2e-llm-inference-service] uid: 67e6f506-8a6c-4c1a-8fe6-4d04246cb0db [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-shadow-service [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-shadow-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '66272' [e2e-llm-inference-service] uid: 7e4b157d-5790-4f3c-9a43-a2ca847d7b1b [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-inference-pool-ip-2bbd07cf.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '66018' [e2e-llm-inference-service] uid: a18852e2-c8a8-4471-86d0-5a321799c126 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1beta1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-scheduler [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '65985' [e2e-llm-inference-service] uid: 67e6f506-8a6c-4c1a-8fe6-4d04246cb0db [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1beta1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-shadow-service [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-shadow-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '66272' [e2e-llm-inference-service] uid: 7e4b157d-5790-4f3c-9a43-a2ca847d7b1b [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-inference-pool-ip-2bbd07cf.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1beta1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '66018' [e2e-llm-inference-service] uid: a18852e2-c8a8-4471-86d0-5a321799c126 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1alpha3 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-scheduler [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '65985' [e2e-llm-inference-service] uid: 67e6f506-8a6c-4c1a-8fe6-4d04246cb0db [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1alpha3 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-shadow-service [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:44Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-shadow-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '66272' [e2e-llm-inference-service] uid: 7e4b157d-5790-4f3c-9a43-a2ca847d7b1b [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-inference-pool-ip-2bbd07cf.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1alpha3 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:24Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '66018' [e2e-llm-inference-service] uid: a18852e2-c8a8-4471-86d0-5a321799c126 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: inference.networking.x-k8s.io/v1alpha2 [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: inference.networking.x-k8s.io/v1alpha2 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e9e92640-abc2-4d67-be0f-4a494428d3bb"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:extensionRef: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureMode: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:portNumber: {} [e2e-llm-inference-service] f:selector: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:targetPortNumber: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-16T18:37:23Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e9e92640-abc2-4d67-be0f-4a494428d3bb [e2e-llm-inference-service] resourceVersion: '65933' [e2e-llm-inference-service] uid: 03792eda-cb6f-4042-ab28-91825006cf25 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] extensionRef: [e2e-llm-inference-service] failureMode: FailOpen [e2e-llm-inference-service] group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] portNumber: 9002 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] targetPortNumber: 8000 [e2e-llm-inference-service] status: [e2e-llm-inference-service] parent: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '1970-01-01T00:00:00Z' [e2e-llm-inference-service] message: Waiting for controller [e2e-llm-inference-service] reason: Pending [e2e-llm-inference-service] status: Unknown [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Status [e2e-llm-inference-service] name: default [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24 [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:56:24Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] pod-template-hash: 74c855b64f [e2e-llm-inference-service] timestamp: '2026-07-16T18:56:13Z' [e2e-llm-inference-service] window: 14.277s [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: llm-d-routing-sidecar [e2e-llm-inference-service] usage: [e2e-llm-inference-service] cpu: 10834138n [e2e-llm-inference-service] memory: 23468Ki [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] usage: [e2e-llm-inference-service] cpu: 18686304n [e2e-llm-inference-service] memory: 24456Ki [e2e-llm-inference-service] apiVersion: metrics.k8s.io/v1beta1 [e2e-llm-inference-service] kind: PodMetrics [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:56:24Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] pod-template-hash: 78b7f4c847 [e2e-llm-inference-service] timestamp: '2026-07-16T18:56:10Z' [e2e-llm-inference-service] window: 10.841s [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] usage: [e2e-llm-inference-service] cpu: 17675952n [e2e-llm-inference-service] memory: 25408Ki [e2e-llm-inference-service] apiVersion: metrics.k8s.io/v1beta1 [e2e-llm-inference-service] kind: PodMetrics [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] creationTimestamp: '2026-07-16T18:56:24Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 6fc5d87465 [e2e-llm-inference-service] timestamp: '2026-07-16T18:56:04Z' [e2e-llm-inference-service] window: 14.9s [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] usage: [e2e-llm-inference-service] cpu: 16659664n [e2e-llm-inference-service] memory: 34976Ki [e2e-llm-inference-service] apiVersion: metrics.k8s.io/v1beta1 [e2e-llm-inference-service] kind: PodMetrics [e2e-llm-inference-service] [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [test_llm_inference_service] [2026-07-16T18:56:25.281119] end - ❌ 1143.450s: Service returned 502: [e2e-llm-inference-service] =============================== warnings summary =============================== [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] [e2e-llm-inference-service] /workspace/source/python/kserve/.venv/lib64/python3.11/site-packages/pytest_asyncio/plugin.py:761: DeprecationWarning: The event_loop fixture provided by pytest-asyncio has been redefined in [e2e-llm-inference-service] /workspace/source/test/e2e/conftest.py:43 [e2e-llm-inference-service] Replacing the event_loop fixture with a custom implementation is deprecated [e2e-llm-inference-service] and will lead to errors in the future. [e2e-llm-inference-service] If you want to request an asyncio event loop with a scope other than function [e2e-llm-inference-service] scope, use the "scope" argument to the asyncio mark when marking the tests. [e2e-llm-inference-service] If you want to return different types of event loops, use the event_loop_policy [e2e-llm-inference-service] fixture. [e2e-llm-inference-service] [e2e-llm-inference-service] warnings.warn( [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator2] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py:40: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_tls.py::test_llm_tls_resources[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_tls.py:93: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-qwen2.5-0.5b] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-custom-template-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-nonzero-threshold-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html [e2e-llm-inference-service] ---------- generated xml file: /workspace/artifacts-dir/junit_e2e.xml ---------- [e2e-llm-inference-service] --------------------------------- JSON report ---------------------------------- [e2e-llm-inference-service] report saved to: /workspace/artifacts-dir/e2e_results.json [e2e-llm-inference-service] =========================== short test summary info ============================ [e2e-llm-inference-service] FAILED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] ====== 1 failed, 41 passed, 3 skipped, 29 warnings in 4091.80s (1:08:11) ======= [must-gather] [must-gather ] OUT 2026-07-16T18:56:26.870887258Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 5189ab8f-0cf7-4a09-8121-908b3bb10fb8 [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.24" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-07-16T18:56:26.928499807Z namespace/openshift-must-gather-wn9zv created [must-gather] [must-gather ] OUT 2026-07-16T18:56:26.936153298Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-p5d8p created [must-gather] [must-gather ] OUT 2026-07-16T18:56:26.962836278Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:33.635469837Z [disk usage checker] Started [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:33.638828092Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:33.900842502Z Error from server (NotFound): namespaces "redhat-ods-operator" not found [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.067507253Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.067540194Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.067540194Z namespaces "redhat-ods-operator" not found [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.069804905Z Error getting logs from redhat-ods-operator [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.239805564Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.239845233Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.239845233Z namespaces "redhat-ods-monitoring" not found [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.242173526Z Error getting logs from redhat-ods-monitoring [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.405551785Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.405583250Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.405583250Z namespaces "redhat-ods-applications" not found [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.407722732Z Error getting logs from redhat-ods-applications [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.569273173Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.569322801Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.569322801Z namespaces "rhods-notebooks" not found [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.571293864Z Error getting logs from rhods-notebooks [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.731943396Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.731972999Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.731972999Z namespaces "rhoai-model-registries" not found [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.733944089Z Error getting logs from rhoai-model-registries [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.896758546Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.896788760Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.896788760Z namespaces "istio-system" not found [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:34.898977328Z Error getting logs from istio-system [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.063364191Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.063394612Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.063394612Z namespaces "knative-serving" not found [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.065717275Z Error getting logs from knative-serving [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.229298955Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.229330411Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.229330411Z namespaces "redhat-ods-applications-auth-provider" not found [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.231381188Z Error getting logs from redhat-ods-applications-auth-provider [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.511719407Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.795374208Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:35.932013303Z error: the server doesn't have a resource type "auths" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.014730430Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.016928731Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.148770044Z error: the server doesn't have a resource type "monitorings" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.233391993Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.235491332Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.365575267Z error: the server doesn't have a resource type "featuretrackers" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.444177760Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.446170842Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.585786980Z error: the server doesn't have a resource type "codeflares" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.666230912Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.668117466Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.802927682Z error: the server doesn't have a resource type "dashboards" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.885062575Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:36.886964569Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.018788854Z error: the server doesn't have a resource type "datasciencepipelines" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.099515853Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.101884479Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.237570488Z error: the server doesn't have a resource type "feastoperators" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.317754850Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.319784666Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.453427473Z error: the server doesn't have a resource type "kserves" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.536786078Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.538888567Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.672226429Z error: the server doesn't have a resource type "kueues" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.753178414Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.755233188Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.885377843Z error: the server doesn't have a resource type "modelcontrollers" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.966512517Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:37.968597922Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.101142879Z error: the server doesn't have a resource type "modelmeshservings" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.180447251Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.182357048Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.313238433Z error: the server doesn't have a resource type "modelregistries" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.395850887Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.397961784Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.532751796Z error: the server doesn't have a resource type "rays" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.611696263Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.613745520Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.643698888Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.745018579Z error: the server doesn't have a resource type "trainingoperators" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.829364284Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.831723986Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:38.961907876Z error: the server doesn't have a resource type "trustyais" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.042408220Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.044707663Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.176008652Z error: the server doesn't have a resource type "workbenches" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.257835545Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.259833347Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.392625723Z error: the server doesn't have a resource type "hardwareprofiles" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.474343962Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.476739550Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.608005886Z error: the server doesn't have a resource type "llamastackoperators" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.692116941Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:39.694363381Z Error collecting info from [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:40.391456682Z error: the server doesn't have a resource type "predictors" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:40.518971930Z error: the server doesn't have a resource type "localmodelnodegroups" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:40.645822925Z error: the server doesn't have a resource type "smcp" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:40.776143878Z error: the server doesn't have a resource type "smm" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:40.904880229Z error: the server doesn't have a resource type "smmr" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:41.489800744Z error: the server doesn't have a resource type "knativeservings" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:41.633379477Z error: the server doesn't have a resource type "configurations" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:41.763954484Z error: the server doesn't have a resource type "routes" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:41.899084219Z error: the server doesn't have a resource type "services" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:42.027702804Z error: the server doesn't have a resource type "revisions" [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:42.560625516Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:42.960216389Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:43.648118837Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:43.669240554Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:44.051049807Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:44.210095497Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:44.563744580Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:45.285728683Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:45.640822361Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:45.807949741Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:46.171732855Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:46.877299101Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:47.249924110Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:47.414142610Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:47.805236539Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:48.520129665Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:48.652491909Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:48.889953910Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:49.048630963Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:49.425095339Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:50.131657922Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:50.506366548Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:50.680825608Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:51.035310877Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:51.753213280Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:52.109684495Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:52.273173394Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:52.667245743Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:53.387790307Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:53.656876940Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:53.774557554Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:53.938660303Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:54.313210046Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:55.089380601Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:55.682451693Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:55.857473799Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:56.236544619Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:56.973887896Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:57.335422634Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:57.497184294Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:57.853723631Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:58.558209297Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:58.662871153Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:58.943667643Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:59.106736563Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:56:59.455979100Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:00.166009124Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:00.530545654Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:00.692482812Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:01.104220930Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:01.811373113Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:02.166696512Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:02.328584095Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:02.680133996Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:03.395893373Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:03.667658606Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:03.777432466Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:03.944046929Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:04.309175753Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:05.030016738Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:05.405479074Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:05.569913092Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:05.933133527Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:06.649989909Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:07.001930479Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:07.169751041Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:07.530650830Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:08.247878879Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:08.628362377Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:08.672069347Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:08.804636451Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:09.398770179Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:09.552597376Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:09.951603893Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:10.117696336Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:10.722246164Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:10.879800868Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:11.270511262Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:11.436615037Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:12.020969425Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:12.167994152Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:12.552224399Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:12.715890674Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:13.321994928Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:13.469685317Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:13.677023220Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:13.852198391Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:14.017224388Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:14.528967713Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:14.718459946Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:15.124165805Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:15.292915861Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:15.837733082Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:16.042269917Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:16.453269559Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:16.620513952Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:17.125545647Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:17.308202368Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:17.708229635Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:17.867786079Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:18.370217305Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:18.548775318Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:18.681677467Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:18.949082066Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:19.111616763Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:19.642050798Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:19.825627418Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:20.232755487Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:20.394592762Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:20.935253454Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:21.116416004Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:21.517250562Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:21.678641659Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:22.189271384Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:22.377723556Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:22.779804046Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:22.943194390Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:23.453407578Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:23.632059680Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:23.687941455Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:24.036566978Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:24.202481183Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:24.711719218Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:24.900412780Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:25.297058344Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:25.457851167Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:25.979960196Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:26.159754215Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:26.557854293Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:26.723547572Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:27.243839464Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:27.426738778Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:27.830091611Z Wrote inspect data to must-gather. [must-gather] [must-gather-zbszc] POD 2026-07-16T18:57:27.855390090Z Caches written to disk [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:28.867965919Z waiting for gather to complete [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:28.872054712Z downloading gather output [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.183510287Z receiving incremental file list [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.201580192Z ./ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.201675218Z aggregated-discovery-api.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.201969375Z aggregated-discovery-apis.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.20305736Z event-filter.html [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.205733229Z timestamp [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.205882368Z version [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206367577Z cluster-scoped-resources/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206386298Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206401729Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.20642075Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206498275Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206508245Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206543397Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206604971Z namespaces/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206614131Z namespaces/kserve-ci-e2e-test/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206657684Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206726408Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.20675398Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206820144Z namespaces/kserve-ci-e2e-test/apps/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206897598Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.206991104Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.207336894Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.207713247Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.207753919Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.207804452Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.207862776Z namespaces/kserve-ci-e2e-test/batch/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.207906658Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.207985043Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.208052917Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.2081073Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.208199356Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.208252119Z namespaces/kserve-ci-e2e-test/core/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.208299592Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.20843776Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.208555607Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.213629379Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.213755176Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.214099687Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.214276067Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.214514521Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.214598856Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.214642309Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.214728414Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.214768936Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.21482151Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.214868882Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.214963258Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.215010981Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.215040803Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.215103956Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.21516349Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.215218633Z namespaces/kserve-ci-e2e-test/pods/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.215227444Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-v8gm9/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.215266406Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-v8gm9/e2e-pvc-model-download-v8gm9.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.215871502Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-v8gm9/storage-initializer/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.215879293Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-v8gm9/storage-initializer/storage-initializer/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.215883233Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-v8gm9/storage-initializer/storage-initializer/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.215924055Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-v8gm9/storage-initializer/storage-initializer/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216073744Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-v8gm9/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216203762Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-v8gm9/storage-initializer/storage-initializer/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216244214Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-mw42d/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216303328Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-mw42d/router-gateway-1-openshift-default-54c47cdfc7-mw42d.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216435626Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-mw42d/istio-proxy/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216443636Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-mw42d/istio-proxy/istio-proxy/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216448127Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-mw42d/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216490159Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-mw42d/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.2166696Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-mw42d/istio-proxy/istio-proxy/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216767405Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-mw42d/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216802797Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-zbq44/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216856361Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-zbq44/router-gateway-2-openshift-default-684dc9b4b9-zbq44.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216959027Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-zbq44/istio-proxy/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216969127Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-zbq44/istio-proxy/istio-proxy/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.216974398Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-zbq44/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217005329Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-zbq44/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217156258Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-zbq44/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217243224Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217287506Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217386292Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/llm-d-routing-sidecar/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217395023Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/llm-d-routing-sidecar/llm-d-routing-sidecar/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217399293Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217476717Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217803007Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217882652Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217901593Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/main/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217915624Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/main/main/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217919734Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/main/main/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.217965027Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/main/main/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218060552Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/main/main/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218141817Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-74c855b64f-5zk24/main/main/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218199811Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218247154Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218325468Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt/main/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218333028Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt/main/main/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218337449Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt/main/main/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218376541Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt/main/main/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218481547Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt/main/main/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218563672Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-2jdrt/main/main/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218582363Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218634777Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k/scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218717091Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k/main/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218727832Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k/main/main/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218733702Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k/main/main/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.218766264Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k/main/main/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219321237Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k/main/main/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219413363Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-6ffwd5k/main/main/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219450175Z namespaces/kserve-ci-e2e-test/policy/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219495338Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.21954256Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219591923Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219639806Z namespaces/kuadrant-system/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219690689Z namespaces/kuadrant-system/kuadrant-system.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219746332Z namespaces/kuadrant-system/apps.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219839638Z namespaces/kuadrant-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219887321Z namespaces/kuadrant-system/apps/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.219932764Z namespaces/kuadrant-system/apps/daemonsets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.220013128Z namespaces/kuadrant-system/apps/deployments.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.22037209Z namespaces/kuadrant-system/apps/replicasets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.220642126Z namespaces/kuadrant-system/apps/statefulsets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.220690689Z namespaces/kuadrant-system/autoscaling/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.220738562Z namespaces/kuadrant-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.220773204Z namespaces/kuadrant-system/batch/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.220817776Z namespaces/kuadrant-system/batch/cronjobs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.220900981Z namespaces/kuadrant-system/batch/jobs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.220954544Z namespaces/kuadrant-system/build.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.221025649Z namespaces/kuadrant-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.221110884Z namespaces/kuadrant-system/build.openshift.io/builds.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.221177538Z namespaces/kuadrant-system/core/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.221238211Z namespaces/kuadrant-system/core/configmaps.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.221422632Z namespaces/kuadrant-system/core/endpoints.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.221531909Z namespaces/kuadrant-system/core/events.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.222038569Z namespaces/kuadrant-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.222118744Z namespaces/kuadrant-system/core/pods.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.222466905Z namespaces/kuadrant-system/core/replicationcontrollers.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.222594242Z namespaces/kuadrant-system/core/secrets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.222807935Z namespaces/kuadrant-system/core/services.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.222930482Z namespaces/kuadrant-system/discovery.k8s.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.222977705Z namespaces/kuadrant-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223095312Z namespaces/kuadrant-system/image.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223140405Z namespaces/kuadrant-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223202708Z namespaces/kuadrant-system/k8s.ovn.org/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223246061Z namespaces/kuadrant-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223333856Z namespaces/kuadrant-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223377219Z namespaces/kuadrant-system/monitoring.coreos.com/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223419201Z namespaces/kuadrant-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223453493Z namespaces/kuadrant-system/networking.k8s.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223511387Z namespaces/kuadrant-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223561799Z namespaces/kuadrant-system/pods/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.22357748Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223604312Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino-7c77957575-zcvdz.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223684027Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223708478Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/authorino/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223713479Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/authorino/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.223722399Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/authorino/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227008804Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/authorino/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227070518Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/authorino/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227113591Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227160524Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/authorino-operator-6d85f6564-5q86f.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227282771Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227300922Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/manager/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227305132Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/manager/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227328014Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/manager/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227541266Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.22760687Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/manager/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227635772Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227681524Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/dns-operator-controller-manager-65b49595d7-7cst2.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.22776971Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.22778002Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/manager/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227785241Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/manager/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227797231Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/manager/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227917689Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.227989433Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/manager/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228020215Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228076478Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin-85fd4b8b9d-htdmf.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228173174Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228184584Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/kuadrant-console-plugin/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228194425Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/kuadrant-console-plugin/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228235588Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/kuadrant-console-plugin/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228312412Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228379096Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228415978Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228466421Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228576388Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228584738Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/manager/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.228588858Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/manager/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.22861938Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/manager/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235012511Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235082265Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/manager/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235101026Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235222653Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador-limitador-69574b596d-xfp6q.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235326359Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.23533506Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/limitador/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.2353401Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/limitador/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235364101Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/limitador/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235455737Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/limitador/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235525861Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/limitador/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235562863Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235596565Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/limitador-operator-controller-manager-6f9f468797-5j59b.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235688941Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235697381Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/manager/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235711272Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/manager/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235740874Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/manager/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235881702Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235949186Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/manager/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.235991159Z namespaces/kuadrant-system/policy/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.2360184Z namespaces/kuadrant-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.236088564Z namespaces/kuadrant-system/route.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.236117646Z namespaces/kuadrant-system/route.openshift.io/routes.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.236189141Z namespaces/openshift-ingress/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.236243194Z namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.236305708Z namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.23635494Z namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.236393253Z namespaces/openshift-ingress/apps/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.236439996Z namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.23652405Z namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.236761455Z namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.23701741Z namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.237076863Z namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.237109965Z namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.237216702Z namespaces/openshift-ingress/batch/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.237265285Z namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.237356Z namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.237401813Z namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.237446365Z namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.23752836Z namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.237576413Z namespaces/openshift-ingress/core/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.237611155Z namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.238046891Z namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.238190509Z namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.238435124Z namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.238547451Z namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.238792996Z namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.238947185Z namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239273014Z namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239338648Z namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.23937177Z namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239465285Z namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239496897Z namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239562001Z namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239613894Z namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239720411Z namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239777994Z namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239836747Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239902131Z namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239943664Z namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.239998277Z namespaces/openshift-ingress/pods/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.240005917Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.240061981Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/istiod-openshift-gateway-94bb8fbfd-wnhkc.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.24020828Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.24022008Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.240224241Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.240275033Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.264949311Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.264994294Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.265009925Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.265075349Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/openshift-ai-inference-openshift-default-866675cbb8-qnxtj.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.265252109Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.265290612Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.265300832Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.265325323Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.266689805Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.266816422Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.266854185Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router-default-7454758546-dtbkw.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.266998943Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.267006844Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.267017794Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.267062897Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/current.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.267219086Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/previous.insecure.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.267295261Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/previous.log [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.267339573Z namespaces/openshift-ingress/policy/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.267383016Z namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.26744292Z namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.267505983Z namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.276775664Z [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.276827618Z sent 3,462 bytes received 1,205,310 bytes 805,848.00 bytes/sec [must-gather] [must-gather-zbszc] OUT 2026-07-16T18:57:29.276847979Z total size is 14,650,686 speedup is 12.12 [must-gather] [must-gather ] OUT 2026-07-16T18:57:29.548659716Z namespace/openshift-must-gather-wn9zv deleted [must-gather] [must-gather] [must-gather] Reprinting Cluster State: [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 5189ab8f-0cf7-4a09-8121-908b3bb10fb8 [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.24" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-07-16T18:57:29.684818644Z Using must-gather plug-in image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ec435553f79197f28a94d55da851b8302383238997d790ed3335149e3ee4341f [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 5189ab8f-0cf7-4a09-8121-908b3bb10fb8 [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.24" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-07-16T18:57:29.70962487Z namespace/openshift-must-gather-kt8tj created [must-gather] [must-gather ] OUT 2026-07-16T18:57:29.71736701Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-9xnfx created [must-gather] [must-gather ] OUT 2026-07-16T18:57:29.740383059Z pod for plug-in image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ec435553f79197f28a94d55da851b8302383238997d790ed3335149e3ee4341f created [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:31.075809162Z [disk usage checker] Started [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:31.079225378Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:31.300136671Z Gathering data for ns/openshift-cluster-version... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:31.474970162Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:31.575737609Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:31.683314736Z Gathering data for ns/default... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:32.054083132Z Gathering data for ns/openshift... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:32.542197782Z Gathering data for ns/kube-system... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.361803811Z Gathering data for ns/openshift-etcd... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.750729682Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.750763348Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.750763348Z namespaces "assisted-installer" not found [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.960128579Z Waiting on subprocesses to finish execution. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.969565414Z Executing Istio gather script [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.972452754Z INFO: Gathering on-disk MachineConfig from degraded nodes [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.978532170Z INFO: Gathering HAProxy config files [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.981684171Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.992300123Z INFO: Waiting for node performance related collection to complete ... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:33.996833306Z INFO: Gathering machine config daemon's old logs from all nodes [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.079423882Z INFO: Collecting host service logs for crio [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.079775955Z INFO: Collecting host service logs for kubelet [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.080124152Z INFO: Collecting host service logs for rpm-ostreed [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.080474976Z INFO: Collecting host service logs for ostree-finalize-staged [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.080827366Z INFO: Collecting host service logs for machine-config-daemon-firstboot [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.081150358Z INFO: Collecting host service logs for machine-config-daemon-host [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.081509755Z INFO: Collecting host service logs for NetworkManager [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.081876132Z INFO: Collecting host service logs for openvswitch [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.082195643Z INFO: Collecting host service logs for ovs-configuration [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.082539713Z INFO: Collecting host service logs for ovsdb-server [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.082878879Z INFO: Collecting host service logs for ovs-vswitchd [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.083275695Z INFO: Waiting for worker host service log collection to complete ... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.681146268Z No resources found in openshift-etcd namespace. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:34.757656564Z INFO: "sriov-network-operator" not detected. Skipping. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.052017895Z INFO: "metallb-operator" not detected. Skipping. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.068052620Z INFO: Waiting for on-disk MachineConfig collection to complete ... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.068129872Z INFO: on-disk MachineConfig config collection complete. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.180621459Z INFO: namespace openshift-frr-k8s not detected. Skipping. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.309905009Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.362486338Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.596460868Z ERROR: No running kube-apiserver pods found [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.642803539Z No resources found [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.711722659Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-psmpz [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.711722659Z insights-runtime-extractor-fsbjf [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.711722659Z insights-runtime-extractor-ftsqx [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.711722659Z insights-runtime-extractor-l9zbv [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.795515574Z error: the server doesn't have a resource type "performanceprofile" [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.909898301Z INFO: Found 1 replicas - prometheus-k8s-0 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:35.977247385Z INFO: Worker host service log collection to complete. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:36.003751201Z INFO: Waiting for HAProxy config collection to complete ... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:36.210897853Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:36.289592251Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:36.289592251Z See 'oc rsync -h' for help and examples [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:36.421172346Z INFO: Getting alertmanagers from prometheus-k8s-0 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:36.582295660Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:36.836859849Z No resources found [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:36.884183481Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.033276213Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.178649788Z Inspecting resource clusterrole.rbac.authorization.k8s.io/istio-reader-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.184076698Z INFO: 'previous-logs' folder not found on ip-10-0-128-180.ec2.internal, skipping... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.278683469Z tar: Removing leading `/' from member names [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.284087828Z INFO: HAProxy config collection complete. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.292729349Z error: the server doesn't have a resource type "multi-networkpolicy" [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.306793193Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.336692366Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.449538596Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.609830808Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.727531208Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.753897355Z INFO: Getting rules from prometheus-k8s-0 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.832157186Z error: the server doesn't have a resource type "machineconfigs" [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.840744810Z INFO: 'previous-logs' folder not found on ip-10-0-132-31.ec2.internal, skipping... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.889305651Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:37.894054600Z Inspecting resource clusterrole.rbac.authorization.k8s.io/istiod-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.218906975Z error: the server doesn't have a resource type "machineconfigpools" [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.228384887Z W0716 18:57:38.228336 1114 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.262363055Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.269142599Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.403352661Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.407724569Z Inspecting resource clusterrole.rbac.authorization.k8s.io/istiod-gateway-controller-openshift-gateway-openshift-ingress [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.481689491Z INFO: 'previous-logs' folder not found on ip-10-0-142-99.ec2.internal, skipping... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.481792164Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.481860656Z INFO: Machine Config Daemon termination log collection complete. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.528029333Z INFO: Getting status/config from prometheus-k8s-0 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.646370907Z Gathering data for ns/openshift-monitoring... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.778698399Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.799070896Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.867598394Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.870224203Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:38.870710677Z Inspecting resource clusterrolebinding.rbac.authorization.k8s.io/istio-reader-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.044185226Z error: the server doesn't have a resource type "kubeletconfigs" [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.154194393Z INFO: Getting status/flags from prometheus-k8s-0 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.192760622Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.195753506Z Inspecting resource clusterrolebinding.rbac.authorization.k8s.io/istiod-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.281432830Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.406056575Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.438316417Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.488619777Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.491553346Z Inspecting resource clusterrolebinding.rbac.authorization.k8s.io/istiod-gateway-controller-openshift-gateway-openshift-ingress [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.683329424Z INFO: INTERCONNECT MODE [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.683359330Z INFO: Gathering ovn-kubernetes DBs [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.714150352Z INFO: Getting targets?state=active from prometheus-k8s-0 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.734548488Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.783589772Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.906650568Z INFO: Gathering OVN_Northbound from ovnkube-node-2vqpm... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.907351169Z INFO: Gathering OVN_Northbound from ovnkube-node-vhthf... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.907722091Z INFO: Gathering OVN_Northbound from ovnkube-node-vm2hw... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.908069174Z INFO: Gathering OVN_Southbound from ovnkube-node-2vqpm... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.908451282Z INFO: Gathering OVN_Southbound from ovnkube-node-vhthf... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:39.910058958Z INFO: Gathering OVN_Southbound from ovnkube-node-vm2hw... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.045258204Z INFO: Getting status/tsdb from prometheus-k8s-0 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.131024071Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.237561728Z tar: Removing leading `/' from member names [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.308326688Z tar: Removing leading `/' from member names [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.311980599Z tar: Removing leading `/' from member names [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.322969477Z tar: Removing leading `/' from member names [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.353484115Z tar: Removing leading `/' from member names [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.389591340Z tar: Removing leading `/' from member names [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.576217291Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.719551674Z INFO: Getting status from alertmanager-main-0 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:40.966637292Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.157936612Z INFO: Waiting for network log collection to complete ... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.157980462Z INFO: Waiting for ovnk database copies to complete ... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.158380969Z INFO: Copying ovnk databases complete. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.160133615Z 7.3M must-gather/network_logs/ovnk_database_store [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.162392456Z ovnk_database_store/ [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.162392456Z ovnk_database_store/ovnkube-node-vhthf_nbdb [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.179560920Z ovnk_database_store/ovnkube-node-vm2hw_nbdb [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.182654823Z W0716 18:57:41.182622 353 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.190045617Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.198330595Z ovnk_database_store/ovnkube-node-vm2hw_sbdb [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.240706467Z ovnk_database_store/ovnkube-node-2vqpm_sbdb [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.256675395Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.283381725Z ovnk_database_store/ovnkube-node-vhthf_sbdb [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.327718297Z ovnk_database_store/ovnkube-node-2vqpm_nbdb [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.345626198Z INFO: Network log collection complete. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.365643754Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.487834404Z Gathering data for ns/openshift-network-console... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:41.719919099Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.007364060Z Gathering data for ns/openshift-console-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.024040451Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.027230784Z INFO: Fallback to identify the container image from release info [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.168764238Z Inspecting resource crd/authorizationpolicies.security.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.349864218Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.483747234Z Gathering data for ns/openshift-console... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.570358367Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.573388828Z Inspecting resource crd/destinationrules.networking.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.603521742Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.742166610Z daemonset.apps/perf-node-gather-daemonset created [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.862031370Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:42.897892438Z Waiting for performance profile collector pods to become ready: 1 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:43.095331798Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:43.098729250Z Inspecting resource crd/envoyfilters.networking.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:43.146064910Z Gathering data for ns/openshift-cluster-storage-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:43.336983552Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:43.563995269Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:43.566157233Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:43.769169033Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:43.796562455Z Gathering data for ns/openshift-dns-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:43.997552122Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:44.001446967Z Inspecting resource crd/gateways.gateway.networking.k8s.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:44.045632325Z Waiting for performance profile collector pods to become ready: 2 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:44.180771836Z Gathering data for ns/openshift-dns... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:44.221931002Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:44.413383555Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:44.415681168Z Inspecting resource crd/gateways.networking.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:44.607019298Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:44.822178525Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:44.826052382Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:44.958039282Z Gathering data for ns/openshift-image-registry... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:45.028084114Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:45.202714617Z Waiting for performance profile collector pods to become ready: 3 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:45.257103812Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:45.259693345Z Inspecting resource crd/httproutes.gateway.networking.k8s.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:45.492455495Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:45.565947460Z Gathering data for ns/openshift-ingress-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:45.728090319Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:45.730445124Z Inspecting resource crd/inferencemodelrewrites.inference.networking.x-k8s.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:45.914063760Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.003034218Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.134919741Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.137615514Z Inspecting resource crd/inferenceobjectives.inference.networking.x-k8s.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.263623794Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.358022980Z Waiting for performance profile collector pods to become ready: 4 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.362384516Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.572345380Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.575025462Z Inspecting resource crd/inferencepoolimports.inference.networking.x-k8s.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.762633396Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.997027057Z Gathering data for ns/openshift-ingress-canary... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:46.999047096Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:47.001612388Z Inspecting resource crd/inferencepools.inference.networking.k8s.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:47.189316879Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:47.402665758Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:47.405612049Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:47.513662881Z Waiting for performance profile collector pods to become ready: 5 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:47.522826708Z Gathering data for ns/openshift-insights... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:47.595849836Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:47.799361972Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:47.802663350Z Inspecting resource crd/peerauthentications.security.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:48.002874014Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:48.223596981Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:48.227012252Z Inspecting resource crd/proxyconfigs.networking.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:48.428402836Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:48.622888408Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:48.626057581Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:48.657077476Z Waiting for performance profile collector pods to become ready: 6 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:48.800206452Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:48.997369718Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:49.001020362Z Inspecting resource crd/requestauthentications.security.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:49.176820615Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:49.383374481Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:49.385556615Z Inspecting resource crd/serviceentries.networking.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:49.571921467Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:49.799143979Z Waiting for performance profile collector pods to become ready: 7 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:49.800045889Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:49.803481985Z Inspecting resource crd/sidecars.networking.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:49.988662115Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:50.193676839Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:50.196195571Z Inspecting resource crd/telemetries.telemetry.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:50.385182938Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:50.534649171Z Gathering data for ns/openshift-lws-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:50.583241372Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:50.585643732Z Inspecting resource crd/virtualservices.networking.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:50.798079482Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:50.948246634Z Daemonset perf-node-gather-daemonset ready 3 out of 3 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.037533352Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.042187715Z Inspecting resource crd/wasmplugins.extensions.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.248469841Z Gathering data for ns/kserve... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.252882125Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.253486688Z Collecting performance related data for node ip-10-0-142-99.ec2.internal [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.262464616Z Collecting performance related data for node ip-10-0-128-180.ec2.internal [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.269197804Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.271455289Z Collecting performance related data for node ip-10-0-132-31.ec2.internal [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.514721098Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.517539966Z Inspecting resource crd/workloadentries.networking.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.736700793Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.988654214Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:51.991144322Z Inspecting resource crd/workloadgroups.networking.istio.io [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:52.223998975Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:52.507869206Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:52.677801288Z Inspecting resource mutatingwebhookconfiguration.admissionregistration.k8s.io/istio-sidecar-injector-openshift-gateway-openshift-ingress [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:52.960106328Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:53.627727891Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:53.789084615Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:53.949333803Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:54.098925326Z Inspecting resource validatingwebhookconfiguration.admissionregistration.k8s.io/istio-validator-openshift-gateway-openshift-ingress [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:54.194154996Z Gathering data for ns/openshift-config... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:54.372890577Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:54.637169106Z Gathering data for ns/openshift-config-managed... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:55.073587209Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:55.213565413Z Gathering data for ns/openshift-kube-apiserver-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:55.280270110Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:55.437956291Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:55.594401697Z Gathering data for ns/openshift-kube-apiserver... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:55.961499564Z Done executing Istio gather script [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:55.963457767Z Gathering data for ns/openshift-kube-controller-manager... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:56.275171284Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:56.325910511Z Gathering data for ns/openshift-kube-controller-manager-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:56.694488870Z Gathering data for ns/openshift-kube-scheduler... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:57.016982681Z Gathering data for ns/openshift-kube-scheduler-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:57.359296324Z Gathering data for ns/openshift-kube-storage-version-migrator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:57.750892090Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:58.117745259Z Gathering data for ns/openshift-user-workload-monitoring... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:57:58.711263909Z Gathering data for ns/openshift-multus... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:00.037526475Z Collecting kubelet logs for node ip-10-0-142-99.ec2.internal [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:00.039782773Z Collecting kubelet logs for node ip-10-0-128-180.ec2.internal [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:00.041595991Z Collecting kubelet logs for node ip-10-0-132-31.ec2.internal [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:00.320379078Z Gathering data for ns/openshift-ovn-kubernetes... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:00.470832475Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-kt8tj namespace [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:00.475717579Z INFO: Node performance data collection complete. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:01.279914108Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:01.792553693Z Gathering data for ns/openshift-host-network... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:02.172718865Z Gathering data for ns/openshift-network-diagnostics... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:02.751672047Z Gathering data for ns/openshift-network-node-identity... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:03.239435543Z Gathering data for ns/openshift-network-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:03.659165703Z Gathering data for ns/openshift-cloud-network-config-controller... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:03.990979658Z Gathering data for ns/openshift-cluster-node-tuning-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:04.523908229Z Gathering data for ns/openshift-apiserver-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:04.856929084Z Gathering data for ns/openshift-apiserver... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:05.329834802Z Gathering data for ns/openshift-controller-manager-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:05.634803194Z Gathering data for ns/openshift-controller-manager... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:05.983556134Z Gathering data for ns/openshift-cluster-samples-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:06.286083109Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:06.670161157Z Gathering data for ns/openshift-operator-lifecycle-manager... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:07.044489591Z Gathering data for ns/openshift-service-ca-operator... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:07.398209277Z Gathering data for ns/openshift-service-ca... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:07.853568014Z Gathering data for ns/openshift-cluster-csi-drivers... [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:08.862858072Z Wrote inspect data to must-gather. [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:08.862920449Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:08.862920449Z skipping gathering secrets/support due to error: secrets "support" not found [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:08.999514463Z error: the server doesn't have a resource type "clusters" [must-gather] [must-gather-xbqkf] POD 2026-07-16T18:58:09.199576678Z Caches written to disk [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.209477362Z waiting for gather to complete [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.213401585Z downloading gather output [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.487477387Z receiving incremental file list [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.510361272Z ./ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.510451708Z aggregated-discovery-api.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.510780878Z aggregated-discovery-apis.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.511525954Z event-filter.html [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.514572322Z timestamp [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.514682249Z version [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.523289091Z cluster-scoped-resources/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.523336314Z cluster-scoped-resources/admissionregistration.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.523388518Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.523492824Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/default-network-annotation.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.523791382Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/servicecidrs.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.523963503Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/user-defined-networks-namespace-label.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.524048018Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.524115393Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/default-network-annotation-binding.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.524252861Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/servicecidrs-binding.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.524351887Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/user-defined-networks-namespace-label-binding.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.524494956Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.524544859Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/multus.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.524681417Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/network-node-identity.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.524763473Z cluster-scoped-resources/apiextensions.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.525223111Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.525292085Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/accounts.nim.opendatahub.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.525492858Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/adminnetworkpolicies.policy.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.525764684Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/adminpolicybasedexternalroutes.k8s.ovn.org.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.525988138Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertingrules.monitoring.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.5261729Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertmanagerconfigs.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.52989725Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertmanagers.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.531477568Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertrelabelconfigs.monitoring.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.531631027Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apikeyapprovals.devportal.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.531767536Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apikeyrequests.devportal.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.531913835Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apikeys.devportal.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.532062654Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apiproducts.devportal.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.532604798Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apirequestcounts.apiserver.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.532772258Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apiservers.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.532942988Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/appliedmanifestworks.work.open-cluster-management.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.533087497Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authconfigs.authorino.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.533906938Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authentications.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.534208037Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorinos.operator.authorino.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.534345225Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorizationpolicies.security.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.534528166Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authpolicies.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.535750742Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/baselineadminnetworkpolicies.policy.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.535995497Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/builds.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.536216601Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/bundles.trust.cert-manager.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.536393252Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/catalogsources.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.536717702Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certificaterequests.cert-manager.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.536874882Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certificates.cert-manager.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.537094125Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certmanagers.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.537395824Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/challenges.acme.cert-manager.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.538080306Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudcredentials.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.538234196Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudeventsources.eventing.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.538367204Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudprivateipconfigs.cloud.network.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.538498682Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterclaims.cluster.open-cluster-management.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.53862607Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustercloudeventsources.eventing.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.538751178Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustercsidrivers.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.538931249Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterimagepolicies.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.539149963Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterissuers.cert-manager.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.539974654Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusteroperators.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.540147954Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterresourcequotas.quota.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.540299534Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterserviceversions.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.542225823Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterstoragecontainers.serving.kserve.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.542401073Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustertriggerauthentications.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.542596726Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusteruserdefinednetworks.k8s.ovn.org.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.542839521Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterversions.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.54380581Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.544373726Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.544491083Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.samples.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.544627031Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleclidownloads.console.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.544739828Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleexternalloglinks.console.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.544853375Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolelinks.console.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.544979373Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolenotifications.console.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.54509038Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleplugins.console.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.545292162Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolequickstarts.console.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.545450662Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoles.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.545554059Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoles.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.545878779Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolesamples.console.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.546016927Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleyamlsamples.console.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.546144085Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/controllerconfigs.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.546823877Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/credentialsrequests.cloudcredential.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.546958186Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/csisnapshotcontrollers.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.547078573Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/datascienceclusters.datasciencecluster.opendatahub.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.547323048Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/destinationrules.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.54816182Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnses.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.548286388Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnses.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.548511722Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnshealthcheckprobes.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.548622949Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnspolicies.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.54880413Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnsrecords.ingress.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.548934208Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnsrecords.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.549202234Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dscinitializations.dscinitialization.opendatahub.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.549352174Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressfirewalls.k8s.ovn.org.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.549477551Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressips.k8s.ovn.org.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.549596499Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressqoses.k8s.ovn.org.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.549739918Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressrouters.network.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.549900757Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressservices.k8s.ovn.org.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.550018905Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/envoyfilters.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.550182815Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/etcds.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.550336164Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/featuregates.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.550470973Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gatewayclasses.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.550627633Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.551050999Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.551253841Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/grpcroutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.551732411Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/helmchartrepositories.helm.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.551851448Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/httproutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.55268672Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/httpscaledobjects.http.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.552833159Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagecontentpolicies.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.552956866Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagecontentsourcepolicies.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.553068363Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagedigestmirrorsets.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.553216613Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagepolicies.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.553429186Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagepruners.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.553678881Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/images.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.553812719Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagetagmirrorsets.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.553927286Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencegraphs.serving.kserve.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.554087747Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencemodelrewrites.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.554234285Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferenceobjectives.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.554376404Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepoolimports.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.554497532Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.554658262Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.554785399Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferenceservices.serving.kserve.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.556982326Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/infrastructures.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.557455885Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ingresscontrollers.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.55803278Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ingresses.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.558225602Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/insightsoperators.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.558373641Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/installplans.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.55851288Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/interceptorroutes.http.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.558647198Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ipamclaims.k8s.cni.cncf.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.558757765Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ippools.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.558860902Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/issuers.cert-manager.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.559688483Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/istiocsrs.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.559975691Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kedacontrollers.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.562820156Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kuadrants.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.562949335Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubeapiservers.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.564446657Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubecontrollermanagers.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.564599497Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubeschedulers.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.564758066Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubestorageversionmigrators.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.564883424Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/leaderworkersetoperators.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.565018932Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/leaderworkersets.leaderworkerset.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.568402512Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/limitadors.limitador.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.568665658Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/llminferenceserviceconfigs.serving.kserve.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.574733413Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/llminferenceservices.serving.kserve.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.579244912Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineconfignodes.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.579399642Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineconfigurations.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.579724052Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineosbuilds.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.579911284Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineosconfigs.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.580059773Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/network-attachment-definitions.k8s.cni.cncf.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.580198261Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/networks.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.580366512Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/networks.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.580625188Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/nodes.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.580735255Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/nodeslicepools.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.580842201Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/oauths.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.581032323Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/oidcpolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.581183182Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/olmconfigs.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.58130662Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/openshiftapiservers.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.581424487Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/openshiftcontrollermanagers.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.581536214Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorconditions.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.581668432Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorgroups.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.581809801Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorhubs.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.58195513Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorpkis.network.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.582063577Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operators.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.582255599Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/orders.acme.cert-manager.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.582420679Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/overlappingrangeipreservations.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.582524785Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/peerauthentications.security.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.582655573Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/pinnedimagesets.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.58276885Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/planpolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.582895598Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/podmonitors.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.583183206Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/podnetworkconnectivitychecks.controlplane.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.583312004Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/probes.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.583550049Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/profiles.tuned.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.583663856Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/projecthelmchartrepositories.helm.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.583778703Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/projects.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.58388555Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/prometheuses.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.585861272Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/prometheusrules.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.585988249Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxies.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.586103897Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxyconfigs.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.586244355Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/rangeallocations.security.internal.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.586368493Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ratelimitpolicies.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.586517922Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/referencegrants.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.586633609Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/requestauthentications.security.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.586781598Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/rolebindingrestrictions.authorization.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.586901316Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/scaledjobs.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.588471783Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/scaledobjects.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.588624152Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/schedulers.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.58874275Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/securitycontextconstraints.security.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.588886849Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servicecas.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.589002786Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/serviceentries.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.589214329Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servicemonitors.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.589472715Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servingruntimes.serving.kserve.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.590006518Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/sidecars.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.590303336Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storages.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.590419683Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storagestates.migration.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.59053004Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storageversionmigrations.migration.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.590647598Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/subscriptions.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.59132681Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetries.telemetry.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.591525902Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetrypolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.591643689Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/thanosrulers.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.593078798Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tlspolicies.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.593343144Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tokenratelimitpolicies.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.593579799Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/trainedmodels.serving.kserve.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.593693136Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/triggerauthentications.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.593855466Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/trustmanagers.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.594153045Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tuneds.tuned.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.594274262Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/userdefinednetworks.k8s.ovn.org.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.594438932Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/virtualservices.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.594824216Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumepopulators.populator.storage.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.594929162Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshotclasses.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.595041989Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshotcontents.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.595272564Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshots.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.595423523Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/wasmplugins.extensions.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.595560592Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadentries.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.59569775Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadgroups.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.595833278Z cluster-scoped-resources/apiregistration.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.596154038Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.596219282Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1..yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.596315248Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.acme.cert-manager.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.596393283Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.admissionregistration.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.596478778Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apiextensions.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.596562134Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apiserver.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.596718923Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apps.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.596817979Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.596901284Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authentication.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.59698677Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authorization.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.597076545Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authorization.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.597204963Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.autoscaling.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.597294259Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.batch.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.597384624Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.build.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.59747558Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cert-manager.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.597563076Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.certificates.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.597659121Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cloud.network.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.597752467Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cloudcredential.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.597845493Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.config.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.597931268Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.console.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.598016973Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.coordination.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.598105949Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.datasciencecluster.opendatahub.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.598220996Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.discovery.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.598399377Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.dscinitialization.opendatahub.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.598484092Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.events.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.598572588Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.flowcontrol.apiserver.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.598660753Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.598744929Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.image.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.598836674Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.598994904Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.inference.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.59908361Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.ingress.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.599198687Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.k8s.cni.cncf.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.599291902Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.k8s.ovn.org.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.599386958Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.599470243Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.leaderworkerset.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.599566339Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.599645934Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.59973956Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.monitoring.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.599825565Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.network.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.599913911Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600003276Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600088701Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.nim.opendatahub.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600217109Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.node.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600303155Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.oauth.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600399921Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600504287Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600573222Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.packages.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600669377Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.policy.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600789255Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.project.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600884001Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.quota.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.600978797Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.rbac.authorization.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.601066902Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.resource.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6011983Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.route.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.601299427Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.samples.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.601388242Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.scheduling.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.601481658Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.internal.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.601567573Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.601658439Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.601754575Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.601848141Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.storage.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.601934946Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.telemetry.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.602029592Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.template.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.602142859Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.tuned.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.602298518Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.user.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.602393354Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.work.open-cluster-management.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.602508371Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.cluster.open-cluster-management.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.602598607Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.controlplane.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.602694463Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.devportal.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.602783038Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.eventing.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.602882534Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.extensions.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.60296937Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.extensions.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.603059856Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.http.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.603183433Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.603265868Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.k8s.cni.cncf.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.603356944Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.60345165Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.603538295Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.limitador.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.603636011Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.migration.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.603725826Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.603822212Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.operator.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.603908038Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.603999234Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.policy.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.604093729Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.serving.kserve.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.604221247Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.telemetry.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.604313313Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.trust.cert-manager.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.604412599Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.604500805Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.60459222Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.604682266Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.serving.kserve.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.604772771Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha3.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.604865217Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.admissionregistration.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.604956523Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.external.metrics.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.60507073Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.60523464Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.helm.openshift.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.605327886Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.http.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.605424942Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.605509677Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.metrics.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.605609433Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.monitoring.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.605698599Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.605794465Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.operator.authorino.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.60588397Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.populator.storage.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.605977086Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.security.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.606064611Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.serving.kserve.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.606177008Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.storage.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.606271204Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta3.authorino.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.60636201Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v2.autoscaling.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.606452965Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v2.operators.coreos.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.606588764Z cluster-scoped-resources/config.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.606643217Z cluster-scoped-resources/config.openshift.io/apiservers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.606836549Z cluster-scoped-resources/config.openshift.io/authentications.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.607024171Z cluster-scoped-resources/config.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.607113456Z cluster-scoped-resources/config.openshift.io/clusterimagepolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.607238074Z cluster-scoped-resources/config.openshift.io/clusteroperators.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.607563054Z cluster-scoped-resources/config.openshift.io/clusterversions.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.607675361Z cluster-scoped-resources/config.openshift.io/consoles.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.607761986Z cluster-scoped-resources/config.openshift.io/dnses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.607864642Z cluster-scoped-resources/config.openshift.io/featuregates.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.608005161Z cluster-scoped-resources/config.openshift.io/imagecontentpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.608087426Z cluster-scoped-resources/config.openshift.io/imagedigestmirrorsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.608230835Z cluster-scoped-resources/config.openshift.io/imagepolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.608315701Z cluster-scoped-resources/config.openshift.io/images.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.608412467Z cluster-scoped-resources/config.openshift.io/imagetagmirrorsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.608530494Z cluster-scoped-resources/config.openshift.io/infrastructures.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.60863388Z cluster-scoped-resources/config.openshift.io/ingresses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.608747727Z cluster-scoped-resources/config.openshift.io/networks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.608840003Z cluster-scoped-resources/config.openshift.io/nodes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.608931379Z cluster-scoped-resources/config.openshift.io/oauths.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.609028385Z cluster-scoped-resources/config.openshift.io/operatorhubs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.60911412Z cluster-scoped-resources/config.openshift.io/projects.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.609293321Z cluster-scoped-resources/config.openshift.io/proxies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.609405498Z cluster-scoped-resources/config.openshift.io/schedulers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.609523985Z cluster-scoped-resources/config.openshift.io/clusteroperators/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.609567418Z cluster-scoped-resources/config.openshift.io/clusteroperators/console.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.609703806Z cluster-scoped-resources/config.openshift.io/clusteroperators/csi-snapshot-controller.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.609831884Z cluster-scoped-resources/config.openshift.io/clusteroperators/dns.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.609955782Z cluster-scoped-resources/config.openshift.io/clusteroperators/image-registry.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.610066479Z cluster-scoped-resources/config.openshift.io/clusteroperators/ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.610204027Z cluster-scoped-resources/config.openshift.io/clusteroperators/insights.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.610316554Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-apiserver.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.610508486Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-controller-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.610628564Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-scheduler.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.610744631Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-storage-version-migrator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.610867608Z cluster-scoped-resources/config.openshift.io/clusteroperators/monitoring.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.610982155Z cluster-scoped-resources/config.openshift.io/clusteroperators/network.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.611159696Z cluster-scoped-resources/config.openshift.io/clusteroperators/node-tuning.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.611286944Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-apiserver.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.611398211Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-controller-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.611502868Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-samples.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.611612084Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager-catalog.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.611717691Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager-packageserver.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.611823527Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.611931354Z cluster-scoped-resources/config.openshift.io/clusteroperators/service-ca.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612039881Z cluster-scoped-resources/config.openshift.io/clusteroperators/storage.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.61218473Z cluster-scoped-resources/config.openshift.io/clusterversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612226252Z cluster-scoped-resources/config.openshift.io/clusterversions/version.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612338479Z cluster-scoped-resources/config.openshift.io/consoles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612384032Z cluster-scoped-resources/config.openshift.io/consoles/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612467957Z cluster-scoped-resources/config.openshift.io/featuregates/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.61251119Z cluster-scoped-resources/config.openshift.io/featuregates/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612593165Z cluster-scoped-resources/config.openshift.io/infrastructures/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612634728Z cluster-scoped-resources/config.openshift.io/infrastructures/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612711502Z cluster-scoped-resources/config.openshift.io/oauths/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612755555Z cluster-scoped-resources/config.openshift.io/oauths/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.61283799Z cluster-scoped-resources/config.openshift.io/proxies/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612879493Z cluster-scoped-resources/config.openshift.io/proxies/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612959438Z cluster-scoped-resources/console.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.612968998Z cluster-scoped-resources/console.openshift.io/consoleplugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.613028502Z cluster-scoped-resources/console.openshift.io/consoleplugins/kuadrant-console-plugin.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.613151009Z cluster-scoped-resources/console.openshift.io/consoleplugins/monitoring-plugin.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.613266907Z cluster-scoped-resources/console.openshift.io/consoleplugins/networking-console-plugin.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.61332298Z cluster-scoped-resources/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.613338531Z cluster-scoped-resources/core/nodes/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.613380674Z cluster-scoped-resources/core/nodes/ip-10-0-128-180.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.613591237Z cluster-scoped-resources/core/nodes/ip-10-0-132-31.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.613732336Z cluster-scoped-resources/core/nodes/ip-10-0-142-99.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.613844923Z cluster-scoped-resources/core/persistentvolumes/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.613893266Z cluster-scoped-resources/core/persistentvolumes/pvc-816e24b0-2442-4458-a346-0c49c6eaace7.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.613985021Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.614044955Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.614089707Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/catch-all.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.614213485Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/exempt.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.614299751Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/global-default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.614383886Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-controller-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.614479962Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-scheduler.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.614564317Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-system-service-accounts.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.614715726Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.614819963Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver-sar.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.614908698Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.615000424Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-authentication-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.615087879Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-controller-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.615209377Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-kube-apiserver-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.615304593Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-monitoring-metrics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.615389328Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-apiserver-sar.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.615480574Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-apiserver.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.615565369Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-server.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.615655264Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/probes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.61574609Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/service-accounts.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.615833535Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-leader-election.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.615954673Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-node-high.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.616045668Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-nodes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.616171156Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.616223759Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/catch-all.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.616330296Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/exempt.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.616412761Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/global-default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.616513828Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/leader-election.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.616610934Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/node-high.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.616705509Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/openshift-control-plane-operators.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.61686997Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/system.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.61703911Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/workload-high.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617177029Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/workload-low.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617271704Z cluster-scoped-resources/gateway.networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617283035Z cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617342789Z cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/openshift-default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617431674Z cluster-scoped-resources/imageregistry.operator.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617450655Z cluster-scoped-resources/imageregistry.operator.openshift.io/configs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617495578Z cluster-scoped-resources/imageregistry.operator.openshift.io/configs/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617589754Z cluster-scoped-resources/imageregistry.operator.openshift.io/imagepruners/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617634947Z cluster-scoped-resources/imageregistry.operator.openshift.io/imagepruners/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617700451Z cluster-scoped-resources/migration.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617723842Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617787296Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/console-plugin-storage-version-migration.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.617901123Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/machineconfiguration-controllerconfig-storage-version-migration.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618060893Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/machineconfiguration-machineconfigpool-storage-version-migration.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618142618Z cluster-scoped-resources/oauth.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6181655Z cluster-scoped-resources/oauth.openshift.io/oauthclients/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618307408Z cluster-scoped-resources/oauth.openshift.io/oauthclients/console.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618392164Z cluster-scoped-resources/operator.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618399384Z cluster-scoped-resources/operator.openshift.io/clustercsidrivers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618453857Z cluster-scoped-resources/operator.openshift.io/clustercsidrivers/ebs.csi.aws.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618543843Z cluster-scoped-resources/operator.openshift.io/consoles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618589566Z cluster-scoped-resources/operator.openshift.io/consoles/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618714504Z cluster-scoped-resources/operator.openshift.io/csisnapshotcontrollers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618751746Z cluster-scoped-resources/operator.openshift.io/csisnapshotcontrollers/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618852022Z cluster-scoped-resources/operator.openshift.io/dnses/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.618895755Z cluster-scoped-resources/operator.openshift.io/dnses/default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.61898441Z cluster-scoped-resources/operator.openshift.io/insightsoperators/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619026983Z cluster-scoped-resources/operator.openshift.io/insightsoperators/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619181593Z cluster-scoped-resources/operator.openshift.io/kubeapiservers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619220805Z cluster-scoped-resources/operator.openshift.io/kubeapiservers/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619309481Z cluster-scoped-resources/operator.openshift.io/kubecontrollermanagers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619353583Z cluster-scoped-resources/operator.openshift.io/kubecontrollermanagers/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619419177Z cluster-scoped-resources/operator.openshift.io/kubeschedulers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.61945948Z cluster-scoped-resources/operator.openshift.io/kubeschedulers/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619544565Z cluster-scoped-resources/operator.openshift.io/kubestorageversionmigrators/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619587118Z cluster-scoped-resources/operator.openshift.io/kubestorageversionmigrators/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619681613Z cluster-scoped-resources/operator.openshift.io/networks/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619724496Z cluster-scoped-resources/operator.openshift.io/networks/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619818272Z cluster-scoped-resources/operator.openshift.io/openshiftapiservers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619872935Z cluster-scoped-resources/operator.openshift.io/openshiftapiservers/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619942329Z cluster-scoped-resources/operator.openshift.io/openshiftcontrollermanagers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.619981632Z cluster-scoped-resources/operator.openshift.io/openshiftcontrollermanagers/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620052066Z cluster-scoped-resources/operator.openshift.io/servicecas/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620114Z cluster-scoped-resources/operator.openshift.io/servicecas/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620200156Z cluster-scoped-resources/operator.openshift.io/storages/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620245848Z cluster-scoped-resources/operator.openshift.io/storages/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620357025Z cluster-scoped-resources/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620364146Z cluster-scoped-resources/operators.coreos.com/olmconfigs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620398028Z cluster-scoped-resources/operators.coreos.com/olmconfigs/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620493034Z cluster-scoped-resources/operators.coreos.com/operators/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620541097Z cluster-scoped-resources/operators.coreos.com/operators/authorino-operator.kuadrant-system.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620669785Z cluster-scoped-resources/operators.coreos.com/operators/dns-operator.kuadrant-system.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620778671Z cluster-scoped-resources/operators.coreos.com/operators/leader-worker-set.openshift-lws-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.620939591Z cluster-scoped-resources/operators.coreos.com/operators/limitador-operator.kuadrant-system.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.621057638Z cluster-scoped-resources/operators.coreos.com/operators/openshift-cert-manager-operator.cert-manager-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.621209888Z cluster-scoped-resources/operators.coreos.com/operators/openshift-custom-metrics-autoscaler-operator.openshift-keda.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.621331366Z cluster-scoped-resources/operators.coreos.com/operators/rhcl-operator.kuadrant-system.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.621430752Z cluster-scoped-resources/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6215578Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.621621013Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/aws-ebs-csi-driver-operator-clusterrolebinding.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.62172357Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cloud-network-config-controller.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.621819596Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cluster-node-tuning-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.621989276Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cluster-storage-operator-role.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.622198319Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/csi-snapshot-controller-operator-clusterrole.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.622313136Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/csi-snapshot-controller-runner-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.622421523Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/metrics-daemon-sa-rolebinding.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.622528489Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-admission-controller-webhook.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.622639807Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-ancillary-tools.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.622751763Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-cluster-readers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.622873231Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-group.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.622998369Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-transient.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.623112176Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-whereabouts.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.623257485Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/network-diagnostics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6233515Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/network-node-identity.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.623441096Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-image-registry-pruner.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.623543722Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-iptables-alerter.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.623637638Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.623740384Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-node-identity-limited.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.623865962Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-node-kube-rbac-proxy.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.623966258Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/registry-registry-role.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.624093766Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.624182062Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/aws-ebs-csi-driver-operator-clusterrole.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6243231Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/cloud-network-config-controller.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.624433057Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/cluster-node-tuning-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.624556765Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/console-extensions-reader.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.624717665Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/console-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.624826622Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/console.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.624931498Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/csi-snapshot-controller-operator-clusterrole.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.625040785Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/helm-chartrepos-viewer.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.625141481Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/metrics-daemon-role.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.62527805Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus-admission-controller-webhook.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.625382566Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus-ancillary-tools.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.625487612Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.625595609Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/net-attach-def-project.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.625695896Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/network-diagnostics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.625801692Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/network-node-identity.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.625904488Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-csi-snapshot-controller-runner.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.626011265Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-iptables-alerter.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.626112721Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-cluster-reader.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.626351616Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.626453682Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-kube-rbac-proxy.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.626557989Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-node-limited.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.626666925Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-udn-editor.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.626771532Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-udn-viewer.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.626874088Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/project-helm-chartrepository-editor.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.626984625Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-admin.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627086551Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-basic-user.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627210529Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-storage-admin.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627316896Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-view.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627423132Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:registry.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627534029Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/whereabouts-cni.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627600533Z cluster-scoped-resources/samples.operator.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627607844Z cluster-scoped-resources/samples.operator.openshift.io/configs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627676418Z cluster-scoped-resources/samples.operator.openshift.io/configs/cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627745972Z cluster-scoped-resources/snapshot.storage.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627754373Z cluster-scoped-resources/snapshot.storage.k8s.io/volumesnapshotclasses/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627803536Z cluster-scoped-resources/snapshot.storage.k8s.io/volumesnapshotclasses/csi-aws-vsc.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627884501Z cluster-scoped-resources/storage.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627891881Z cluster-scoped-resources/storage.k8s.io/csidrivers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.627948425Z cluster-scoped-resources/storage.k8s.io/csidrivers/ebs.csi.aws.com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.62802813Z cluster-scoped-resources/storage.k8s.io/csinodes/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628075712Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-128-180.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628219332Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-132-31.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628326098Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-142-99.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628401743Z cluster-scoped-resources/storage.k8s.io/storageclasses/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628476477Z cluster-scoped-resources/storage.k8s.io/storageclasses/gp2-csi.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628605765Z cluster-scoped-resources/storage.k8s.io/storageclasses/gp3-csi.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6286787Z host_service_logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628688661Z host_service_logs/masters/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628748634Z host_service_logs/masters/NetworkManager_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628831609Z host_service_logs/masters/crio_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628906874Z host_service_logs/masters/kubelet_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.628986649Z host_service_logs/masters/machine-config-daemon-firstboot_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629145489Z host_service_logs/masters/machine-config-daemon-host_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629301128Z host_service_logs/masters/openvswitch_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629384253Z host_service_logs/masters/ostree-finalize-staged_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629469389Z host_service_logs/masters/ovs-configuration_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629556714Z host_service_logs/masters/ovs-vswitchd_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629638259Z host_service_logs/masters/ovsdb-server_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629721124Z host_service_logs/masters/rpm-ostreed_service.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629783098Z ingress_controllers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629790409Z ingress_controllers/default/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629796509Z ingress_controllers/default/router-default-7454758546-dtbkw/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629858863Z ingress_controllers/default/router-default-7454758546-dtbkw/haproxy.config [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.629996081Z insights-data/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.630020473Z istio/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.630078616Z istio/aggregated-discovery-api.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.630219945Z istio/aggregated-discovery-apis.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.630716766Z istio/event-filter.html [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.630962781Z istio/timestamp [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.631154283Z istio/cluster-scoped-resources/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.631170164Z istio/cluster-scoped-resources/admissionregistration.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.631180535Z istio/cluster-scoped-resources/admissionregistration.k8s.io/mutatingwebhookconfigurations/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.631231708Z istio/cluster-scoped-resources/admissionregistration.k8s.io/mutatingwebhookconfigurations/istio-sidecar-injector-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.631341825Z istio/cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.631393468Z istio/cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/istio-validator-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.631474083Z istio/cluster-scoped-resources/apiextensions.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.631599721Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.631656964Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorizationpolicies.security.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.631862437Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/destinationrules.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.632772423Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/envoyfilters.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.633033229Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gatewayclasses.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.633288255Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.633826158Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.634041481Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/grpcroutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.634613887Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/httproutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.635648251Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencemodelrewrites.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.635812851Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferenceobjectives.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.63595616Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepoolimports.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.636154832Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.636386377Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.636537796Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/peerauthentications.security.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.636698646Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxyconfigs.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.636833714Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/referencegrants.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.636952152Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/requestauthentications.security.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.637097541Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/serviceentries.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.637311334Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/sidecars.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6375703Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetries.telemetry.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.637787043Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/virtualservices.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.63821666Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/wasmplugins.extensions.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.638365489Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadentries.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.638503708Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadgroups.networking.istio.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.638642936Z istio/cluster-scoped-resources/gateway.networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.638658587Z istio/cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.63870273Z istio/cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/openshift-default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.638778755Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.638787125Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.638833468Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/istio-reader-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.638922883Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/istiod-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639072453Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/istiod-gateway-controller-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.63919287Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639241853Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/istio-reader-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639333979Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/istiod-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639486118Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/istiod-gateway-controller-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639553342Z istio/namespaces/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639564053Z istio/namespaces/kserve-ci-e2e-test/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639573284Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639577274Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639632498Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/router-gateway-1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639736984Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/router-gateway-2.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639824939Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639857121Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/router-route-1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.639960298Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/router-route-2.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640042993Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/router-route-3.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640176471Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/router-route-4.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640279807Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/scheduler-v06-pd-migration-test-kserve-route.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640411826Z istio/namespaces/kserve-ci-e2e-test/inference.networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640419106Z istio/namespaces/kserve-ci-e2e-test/inference.networking.k8s.io/inferencepools/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640469209Z istio/namespaces/kserve-ci-e2e-test/inference.networking.k8s.io/inferencepools/scheduler-v06-pd-migration-test-inference-pool.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640526503Z istio/namespaces/kserve-ci-e2e-test/inference.networking.x-k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640543344Z istio/namespaces/kserve-ci-e2e-test/inference.networking.x-k8s.io/inferencepools/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640550984Z istio/namespaces/kserve-ci-e2e-test/inference.networking.x-k8s.io/inferencepools/scheduler-v06-pd-migration-test-inference-pool.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640619668Z istio/namespaces/kserve-ci-e2e-test/networking.istio.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.64063911Z istio/namespaces/kserve-ci-e2e-test/networking.istio.io/destinationrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640687893Z istio/namespaces/kserve-ci-e2e-test/networking.istio.io/destinationrules/scheduler-v06-pd-migration-test-kserve-scheduler.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640780958Z istio/namespaces/kserve-ci-e2e-test/networking.istio.io/destinationrules/scheduler-v06-pd-migration-test-kserve-shadow-svc.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640926598Z istio/namespaces/kserve-ci-e2e-test/networking.istio.io/destinationrules/scheduler-v06-pd-migration-test-kserve-workload-svc.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.640998582Z istio/namespaces/openshift-ingress/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.641049435Z istio/namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.641109169Z istio/namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.641191854Z istio/namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.641254118Z istio/namespaces/openshift-ingress/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.641303801Z istio/namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.641382226Z istio/namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.641715906Z istio/namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.641954681Z istio/namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.642019175Z istio/namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.642147863Z istio/namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.642325704Z istio/namespaces/openshift-ingress/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.642359856Z istio/namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.642487344Z istio/namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.642561159Z istio/namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.642613222Z istio/namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.64275025Z istio/namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.642845756Z istio/namespaces/openshift-ingress/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.642896329Z istio/namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.643362558Z istio/namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.643465804Z istio/namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.643823607Z istio/namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.643905262Z istio/namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.644241343Z istio/namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.64436511Z istio/namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.644560222Z istio/namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.644645728Z istio/namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.64468886Z istio/namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.644766385Z istio/namespaces/openshift-ingress/gateway.networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.644775045Z istio/namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.644809748Z istio/namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/openshift-ai-inference.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.644891563Z istio/namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.644931095Z istio/namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.64500728Z istio/namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645049062Z istio/namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645149899Z istio/namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645252955Z istio/namespaces/openshift-ingress/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645296158Z istio/namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645359902Z istio/namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645419635Z istio/namespaces/openshift-ingress/networking.istio.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645433056Z istio/namespaces/openshift-ingress/networking.istio.io/envoyfilters/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645478289Z istio/namespaces/openshift-ingress/networking.istio.io/envoyfilters/kuadrant-auth-openshift-ai-inference.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645574235Z istio/namespaces/openshift-ingress/networking.istio.io/envoyfilters/kuadrant-openshift-ai-inference.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645688492Z istio/namespaces/openshift-ingress/networking.istio.io/envoyfilters/openshift-ai-inference-authn-ssl.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645799649Z istio/namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645838431Z istio/namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645914746Z istio/namespaces/openshift-ingress/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645929927Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.645978Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/istiod-openshift-gateway-94bb8fbfd-wnhkc.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.646087697Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.646095387Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.646108628Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.646201104Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.659486866Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.659600653Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.659660976Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.65971645Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/openshift-ai-inference-openshift-default-866675cbb8-qnxtj.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.659814856Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.659830427Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.659837977Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.659907861Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661004759Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661091225Z istio/namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661118116Z istio/namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router-default-7454758546-dtbkw.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661280116Z istio/namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661293547Z istio/namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661300307Z istio/namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.66134317Z istio/namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661455717Z istio/namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661518811Z istio/namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661566334Z istio/namespaces/openshift-ingress/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661600646Z istio/namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.66165384Z istio/namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661689592Z istio/namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661734205Z machine_config_ondisk/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661744685Z machine_config_termination_logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661755596Z monitoring/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661762756Z monitoring/alertmanager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661808779Z monitoring/alertmanager/status.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661900905Z monitoring/alertmanager/status.stderr [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.661962969Z monitoring/prometheus/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.662008862Z monitoring/prometheus/alertmanagers.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.662092487Z monitoring/prometheus/alertmanagers.stderr [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.662187153Z monitoring/prometheus/rules.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.662946749Z monitoring/prometheus/rules.stderr [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.663032045Z monitoring/prometheus/prometheus-k8s-0/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.663165923Z monitoring/prometheus/prometheus-k8s-0/active-targets.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.664042117Z monitoring/prometheus/prometheus-k8s-0/active-targets.stderr [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.664118292Z monitoring/prometheus/prometheus-k8s-0/status/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.664202707Z monitoring/prometheus/prometheus-k8s-0/status/runtimeinfo.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.664296673Z monitoring/prometheus/prometheus-k8s-0/status/runtimeinfo.stderr [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.664364377Z monitoring/prometheus/prometheus-k8s-0/status/tsdb.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.664471554Z monitoring/prometheus/prometheus-k8s-0/status/tsdb.stderr [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.664506926Z monitoring/prometheus/status/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.664559129Z monitoring/prometheus/status/config.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.665023898Z monitoring/prometheus/status/config.stderr [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.665098402Z monitoring/prometheus/status/flags.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.66522511Z monitoring/prometheus/status/flags.stderr [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.665460105Z namespaces/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.665473306Z namespaces/cert-manager-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.665479816Z namespaces/cert-manager-operator/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.665485657Z namespaces/cert-manager-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.665505158Z namespaces/cert-manager-operator/coordination.k8s.io/leases/cert-manager-operator-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.665603074Z namespaces/cert-manager-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.665614954Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.665663588Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6658641Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.666087284Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.666392873Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.666628247Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.666893773Z namespaces/cert-manager-operator/operators.coreos.com/installplans/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.666939506Z namespaces/cert-manager-operator/operators.coreos.com/installplans/install-rv4lz.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667087216Z namespaces/cert-manager-operator/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667148479Z namespaces/cert-manager-operator/operators.coreos.com/operatorconditions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667259526Z namespaces/cert-manager-operator/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667277637Z namespaces/cert-manager-operator/operators.coreos.com/operatorgroups/openshift-cert-manager-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667366373Z namespaces/cert-manager-operator/operators.coreos.com/subscriptions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667416386Z namespaces/cert-manager-operator/operators.coreos.com/subscriptions/openshift-cert-manager-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667510262Z namespaces/cert-manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667522132Z namespaces/cert-manager/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667532743Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667584356Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667752517Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.667937988Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.668112439Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.668371585Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.66860619Z namespaces/default/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.668662313Z namespaces/default/default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.668750919Z namespaces/default/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.668806852Z namespaces/default/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.668900258Z namespaces/default/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.668952781Z namespaces/default/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669050717Z namespaces/default/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669157844Z namespaces/default/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669286921Z namespaces/default/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669323254Z namespaces/default/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669361556Z namespaces/default/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.66942367Z namespaces/default/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669472273Z namespaces/default/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669564229Z namespaces/default/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669643684Z namespaces/default/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669702107Z namespaces/default/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669793883Z namespaces/default/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669891169Z namespaces/default/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.669943352Z namespaces/default/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6700627Z namespaces/default/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.670191048Z namespaces/default/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.670366468Z namespaces/default/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.670457944Z namespaces/default/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.67055342Z namespaces/default/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.670700689Z namespaces/default/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.670871859Z namespaces/default/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.670952455Z namespaces/default/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671017448Z namespaces/default/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671105464Z namespaces/default/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.67120258Z namespaces/default/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671290665Z namespaces/default/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671345159Z namespaces/default/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671445195Z namespaces/default/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671537881Z namespaces/default/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671599565Z namespaces/default/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671652058Z namespaces/default/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671711571Z namespaces/default/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671771435Z namespaces/default/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671826959Z namespaces/default/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671834269Z namespaces/default/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.671893283Z namespaces/default/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.672056573Z namespaces/default/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.672270736Z namespaces/default/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.672419975Z namespaces/default/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.672637049Z namespaces/default/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.672838901Z namespaces/default/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.672872103Z namespaces/default/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.672942568Z namespaces/default/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.67297498Z namespaces/default/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673045664Z namespaces/kserve-ci-e2e-test/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673054205Z namespaces/kserve-ci-e2e-test/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673059065Z namespaces/kserve-ci-e2e-test/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673109658Z namespaces/kserve-ci-e2e-test/coordination.k8s.io/leases/epp-kserve-ci-e2e-test-scheduler-ha-replicas-test-inference-pool.llm-d.ai.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673220205Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673233796Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673279448Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/router-gateway-1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673390695Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/router-gateway-2.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6734702Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673478861Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/podmonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673538835Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/podmonitors/kserve-llm-isvc-vllm-engine-default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673643131Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/podmonitors/kserve-llm-isvc-vllm-engine.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673706615Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673760048Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors/kserve-llm-isvc-scheduler-default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673843003Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors/kserve-llm-isvc-scheduler.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673903657Z namespaces/kserve-ci-e2e-test/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673915938Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.673980862Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.674143182Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.674346045Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.674488473Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.674703187Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.674942951Z namespaces/kserve/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.674999035Z namespaces/kserve/kserve.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.67508291Z namespaces/kserve/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.675139743Z namespaces/kserve/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.67525261Z namespaces/kserve/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.675287813Z namespaces/kserve/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.675376998Z namespaces/kserve/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.675580831Z namespaces/kserve/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.675815375Z namespaces/kserve/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.675878159Z namespaces/kserve/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.675928822Z namespaces/kserve/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.675989766Z namespaces/kserve/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.67604868Z namespaces/kserve/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.676225131Z namespaces/kserve/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.676296165Z namespaces/kserve/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.676356129Z namespaces/kserve/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.676480586Z namespaces/kserve/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.676527299Z namespaces/kserve/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.6765378Z namespaces/kserve/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.676568922Z namespaces/kserve/coordination.k8s.io/leases/kserve-controller-manager-leader-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.676716041Z namespaces/kserve/coordination.k8s.io/leases/llminferenceservice-kserve-controller-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.676809167Z namespaces/kserve/coordination.k8s.io/leases/odh-model-controller.opendatahub.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.676947455Z namespaces/kserve/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.676967116Z namespaces/kserve/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.677269915Z namespaces/kserve/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.677566484Z namespaces/kserve/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.677910105Z namespaces/kserve/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.67798544Z namespaces/kserve/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.678334891Z namespaces/kserve/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.67848017Z namespaces/kserve/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.67879452Z namespaces/kserve/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.678915667Z namespaces/kserve/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.678984521Z namespaces/kserve/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.67911898Z namespaces/kserve/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.679188474Z namespaces/kserve/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.679271299Z namespaces/kserve/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.679332373Z namespaces/kserve/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.679437369Z namespaces/kserve/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.679539856Z namespaces/kserve/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.679627681Z namespaces/kserve/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.679711726Z namespaces/kserve/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.679795111Z namespaces/kserve/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.679841554Z namespaces/kserve/monitoring.coreos.com/servicemonitors/model-serving-api-metrics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.679957492Z namespaces/kserve/monitoring.coreos.com/servicemonitors/odh-model-controller-metrics-monitor.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.680023906Z namespaces/kserve/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.680088Z namespaces/kserve/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.680205767Z namespaces/kserve/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.680220358Z namespaces/kserve/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.680303143Z namespaces/kserve/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.680498305Z namespaces/kserve/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.680695657Z namespaces/kserve/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.68090321Z namespaces/kserve/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.681192708Z namespaces/kserve/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.681432973Z namespaces/kserve/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.681445213Z namespaces/kserve/pods/kserve-controller-manager-576d7b7757-pddx2/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.681497107Z namespaces/kserve/pods/kserve-controller-manager-576d7b7757-pddx2/kserve-controller-manager-576d7b7757-pddx2.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.681618184Z namespaces/kserve/pods/kserve-controller-manager-576d7b7757-pddx2/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.681627655Z namespaces/kserve/pods/kserve-controller-manager-576d7b7757-pddx2/manager/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.681631655Z namespaces/kserve/pods/kserve-controller-manager-576d7b7757-pddx2/manager/manager/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.681694239Z namespaces/kserve/pods/kserve-controller-manager-576d7b7757-pddx2/manager/manager/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.681844288Z namespaces/kserve/pods/kserve-controller-manager-576d7b7757-pddx2/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.681935274Z namespaces/kserve/pods/kserve-controller-manager-576d7b7757-pddx2/manager/manager/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.682010208Z namespaces/kserve/pods/llmisvc-controller-manager-7bc4499b55-c2rg5/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.682047301Z namespaces/kserve/pods/llmisvc-controller-manager-7bc4499b55-c2rg5/llmisvc-controller-manager-7bc4499b55-c2rg5.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.68219305Z namespaces/kserve/pods/llmisvc-controller-manager-7bc4499b55-c2rg5/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.682214031Z namespaces/kserve/pods/llmisvc-controller-manager-7bc4499b55-c2rg5/manager/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.682223842Z namespaces/kserve/pods/llmisvc-controller-manager-7bc4499b55-c2rg5/manager/manager/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.682268124Z namespaces/kserve/pods/llmisvc-controller-manager-7bc4499b55-c2rg5/manager/manager/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.68542793Z namespaces/kserve/pods/llmisvc-controller-manager-7bc4499b55-c2rg5/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.685497104Z namespaces/kserve/pods/llmisvc-controller-manager-7bc4499b55-c2rg5/manager/manager/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.685555018Z namespaces/kserve/pods/model-serving-api-fd65d7d6-29xh8/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.685576149Z namespaces/kserve/pods/model-serving-api-fd65d7d6-29xh8/model-serving-api-fd65d7d6-29xh8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.685696856Z namespaces/kserve/pods/model-serving-api-fd65d7d6-29xh8/server/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.685711457Z namespaces/kserve/pods/model-serving-api-fd65d7d6-29xh8/server/server/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.685721008Z namespaces/kserve/pods/model-serving-api-fd65d7d6-29xh8/server/server/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.68574691Z namespaces/kserve/pods/model-serving-api-fd65d7d6-29xh8/server/server/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.685854726Z namespaces/kserve/pods/model-serving-api-fd65d7d6-29xh8/server/server/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.68591382Z namespaces/kserve/pods/model-serving-api-fd65d7d6-29xh8/server/server/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.685944102Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-w7gqq/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.685986585Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-w7gqq/odh-model-controller-84b54b7f97-w7gqq.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.686063919Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-w7gqq/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.68607519Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-w7gqq/manager/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.686089251Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-w7gqq/manager/manager/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.686142914Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-w7gqq/manager/manager/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707170795Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-w7gqq/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707244309Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-w7gqq/manager/manager/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707285542Z namespaces/kserve/pods/s3-init-4sjxq/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707336295Z namespaces/kserve/pods/s3-init-4sjxq/s3-init-4sjxq.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707431161Z namespaces/kserve/pods/s3-init-4sjxq/download-hf-model/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707441121Z namespaces/kserve/pods/s3-init-4sjxq/download-hf-model/download-hf-model/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707445842Z namespaces/kserve/pods/s3-init-4sjxq/download-hf-model/download-hf-model/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707479774Z namespaces/kserve/pods/s3-init-4sjxq/download-hf-model/download-hf-model/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707566619Z namespaces/kserve/pods/s3-init-4sjxq/download-hf-model/download-hf-model/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707641334Z namespaces/kserve/pods/s3-init-4sjxq/download-hf-model/download-hf-model/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707663995Z namespaces/kserve/pods/s3-init-4sjxq/s3-init/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707674026Z namespaces/kserve/pods/s3-init-4sjxq/s3-init/s3-init/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707720629Z namespaces/kserve/pods/s3-init-4sjxq/s3-init/s3-init/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.707768182Z namespaces/kserve/pods/s3-init-4sjxq/s3-init/s3-init/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708148585Z namespaces/kserve/pods/s3-init-4sjxq/s3-init/s3-init/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708256662Z namespaces/kserve/pods/s3-init-4sjxq/s3-init/s3-init/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708283473Z namespaces/kserve/pods/seaweedfs-64568bcd49-tnr46/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708370089Z namespaces/kserve/pods/seaweedfs-64568bcd49-tnr46/seaweedfs-64568bcd49-tnr46.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708501197Z namespaces/kserve/pods/seaweedfs-64568bcd49-tnr46/seaweedfs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708519288Z namespaces/kserve/pods/seaweedfs-64568bcd49-tnr46/seaweedfs/seaweedfs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.7085478Z namespaces/kserve/pods/seaweedfs-64568bcd49-tnr46/seaweedfs/seaweedfs/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708589283Z namespaces/kserve/pods/seaweedfs-64568bcd49-tnr46/seaweedfs/seaweedfs/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708771954Z namespaces/kserve/pods/seaweedfs-64568bcd49-tnr46/seaweedfs/seaweedfs/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708841568Z namespaces/kserve/pods/seaweedfs-64568bcd49-tnr46/seaweedfs/seaweedfs/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708863129Z namespaces/kserve/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708920723Z namespaces/kserve/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.708994698Z namespaces/kserve/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.709020449Z namespaces/kserve/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.709144617Z namespaces/kuadrant-system/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.70918774Z namespaces/kuadrant-system/kuadrant-system.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.709329488Z namespaces/kuadrant-system/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.709349999Z namespaces/kuadrant-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.709418314Z namespaces/kuadrant-system/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.709463947Z namespaces/kuadrant-system/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.709550382Z namespaces/kuadrant-system/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.70983657Z namespaces/kuadrant-system/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710103786Z namespaces/kuadrant-system/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710183221Z namespaces/kuadrant-system/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710225324Z namespaces/kuadrant-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710302809Z namespaces/kuadrant-system/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710355382Z namespaces/kuadrant-system/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710435687Z namespaces/kuadrant-system/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.71048385Z namespaces/kuadrant-system/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710502971Z namespaces/kuadrant-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710626558Z namespaces/kuadrant-system/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710664471Z namespaces/kuadrant-system/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710672011Z namespaces/kuadrant-system/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710701803Z namespaces/kuadrant-system/coordination.k8s.io/leases/3745a16e.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710797339Z namespaces/kuadrant-system/coordination.k8s.io/leases/a3f98d6c.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.710893435Z namespaces/kuadrant-system/coordination.k8s.io/leases/aac3a15d.authorino.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.71098268Z namespaces/kuadrant-system/coordination.k8s.io/leases/f139389e.kuadrant.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.711000242Z namespaces/kuadrant-system/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.711079997Z namespaces/kuadrant-system/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.711330972Z namespaces/kuadrant-system/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.71145044Z namespaces/kuadrant-system/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.711866315Z namespaces/kuadrant-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.71194405Z namespaces/kuadrant-system/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.712313173Z namespaces/kuadrant-system/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.712442761Z namespaces/kuadrant-system/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.712661314Z namespaces/kuadrant-system/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713024607Z namespaces/kuadrant-system/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.71307536Z namespaces/kuadrant-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713250821Z namespaces/kuadrant-system/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713287973Z namespaces/kuadrant-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713380689Z namespaces/kuadrant-system/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713445073Z namespaces/kuadrant-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713539259Z namespaces/kuadrant-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713617483Z namespaces/kuadrant-system/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713667306Z namespaces/kuadrant-system/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713708029Z namespaces/kuadrant-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713763752Z namespaces/kuadrant-system/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713810715Z namespaces/kuadrant-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713876219Z namespaces/kuadrant-system/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.71388723Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.713926313Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.714144486Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.714393531Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.714539371Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.714768355Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.714980728Z namespaces/kuadrant-system/operators.coreos.com/installplans/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.71501224Z namespaces/kuadrant-system/operators.coreos.com/installplans/install-5qvjp.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.715405104Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.715454687Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.715567834Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.71566183Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.715742415Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.715790448Z namespaces/kuadrant-system/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.715848121Z namespaces/kuadrant-system/operators.coreos.com/operatorgroups/kuadrant.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.715915225Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.715969179Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/authorino-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.716066755Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/dns-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.716286359Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/limitador-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.716390585Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/rhcl-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.71647631Z namespaces/kuadrant-system/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.716484731Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.716512293Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino-7c77957575-zcvdz.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.716602578Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.716609899Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/authorino/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.716616739Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/authorino/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.716676993Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/authorino/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.72019308Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/authorino/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.720263914Z namespaces/kuadrant-system/pods/authorino-7c77957575-zcvdz/authorino/authorino/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.720317368Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.72036174Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/authorino-operator-6d85f6564-5q86f.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.720499059Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.72050793Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.72051462Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/manager/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.720593465Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/manager/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.720797667Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.720864922Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-5q86f/manager/manager/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.720925146Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.720955527Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/dns-operator-controller-manager-65b49595d7-7cst2.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721068184Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721096056Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721102937Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/manager/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721118438Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/manager/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721265836Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.72132936Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-7cst2/manager/manager/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721345461Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721397695Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin-85fd4b8b9d-htdmf.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.72148001Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.72148826Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/kuadrant-console-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721492701Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/kuadrant-console-plugin/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721532873Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/kuadrant-console-plugin/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721596367Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721667001Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-htdmf/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721700393Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721735325Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721852603Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721862964Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721867064Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/manager/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.721879905Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/manager/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727392926Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727451439Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-vt8bd/manager/manager/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727507693Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727523663Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador-limitador-69574b596d-xfp6q.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727672393Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727688174Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/limitador/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727694284Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/limitador/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727734377Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/limitador/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727838003Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/limitador/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727934679Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-xfp6q/limitador/limitador/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.727969211Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728015934Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/limitador-operator-controller-manager-6f9f468797-5j59b.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728158403Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728183484Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728194695Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/manager/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728244348Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/manager/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728390567Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728456731Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-5j59b/manager/manager/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728475393Z namespaces/kuadrant-system/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728522285Z namespaces/kuadrant-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728571158Z namespaces/kuadrant-system/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728610571Z namespaces/kuadrant-system/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728658154Z namespaces/kube-node-lease/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728669574Z namespaces/kube-node-lease/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728681665Z namespaces/kube-node-lease/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728725978Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-128-180.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728826444Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-132-31.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728906709Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-142-99.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728949032Z namespaces/kube-node-lease/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.728970393Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.729017626Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.729261091Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.729462423Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.729607043Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.729819376Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.730032089Z namespaces/kube-public/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.73005126Z namespaces/kube-public/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.73005769Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.730100933Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.73037765Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.730578323Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.730712691Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.730915483Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.73118704Z namespaces/kube-system/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731223483Z namespaces/kube-system/kube-system.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731295097Z namespaces/kube-system/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731338139Z namespaces/kube-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731385133Z namespaces/kube-system/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731415054Z namespaces/kube-system/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731552483Z namespaces/kube-system/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731628068Z namespaces/kube-system/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731715093Z namespaces/kube-system/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731754195Z namespaces/kube-system/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731793548Z namespaces/kube-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731840341Z namespaces/kube-system/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731883283Z namespaces/kube-system/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.731960938Z namespaces/kube-system/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.732011321Z namespaces/kube-system/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.732056934Z namespaces/kube-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.732193602Z namespaces/kube-system/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.732223664Z namespaces/kube-system/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.732259567Z namespaces/kube-system/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.73230825Z namespaces/kube-system/coordination.k8s.io/leases/apiserver-atth65l7hrqanmseejsk7d5tue.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.732399065Z namespaces/kube-system/coordination.k8s.io/leases/cert-manager-cainjector-leader-election.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.73248354Z namespaces/kube-system/coordination.k8s.io/leases/cert-manager-controller.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.732562975Z namespaces/kube-system/coordination.k8s.io/leases/kube-controller-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.732726935Z namespaces/kube-system/coordination.k8s.io/leases/kube-scheduler.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.73279784Z namespaces/kube-system/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.732839322Z namespaces/kube-system/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.733049525Z namespaces/kube-system/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.733304801Z namespaces/kube-system/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.733562297Z namespaces/kube-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.733641972Z namespaces/kube-system/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.733917749Z namespaces/kube-system/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734042537Z namespaces/kube-system/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734377067Z namespaces/kube-system/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734449522Z namespaces/kube-system/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734494135Z namespaces/kube-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.73457453Z namespaces/kube-system/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734622003Z namespaces/kube-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734674646Z namespaces/kube-system/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734710128Z namespaces/kube-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734799614Z namespaces/kube-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734868508Z namespaces/kube-system/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734921021Z namespaces/kube-system/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.734957773Z namespaces/kube-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.735007017Z namespaces/kube-system/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.735027198Z namespaces/kube-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.735091442Z namespaces/kube-system/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.735102382Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.735173047Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.735330126Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.735512308Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.735649896Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.735844208Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736053611Z namespaces/kube-system/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736083913Z namespaces/kube-system/pods/global-pull-secret-syncer-785gt/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736184699Z namespaces/kube-system/pods/global-pull-secret-syncer-785gt/global-pull-secret-syncer-785gt.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736261824Z namespaces/kube-system/pods/global-pull-secret-syncer-785gt/global-pull-secret-syncer/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736273855Z namespaces/kube-system/pods/global-pull-secret-syncer-785gt/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736281165Z namespaces/kube-system/pods/global-pull-secret-syncer-785gt/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736319148Z namespaces/kube-system/pods/global-pull-secret-syncer-785gt/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736714842Z namespaces/kube-system/pods/global-pull-secret-syncer-785gt/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736782726Z namespaces/kube-system/pods/global-pull-secret-syncer-785gt/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736801468Z namespaces/kube-system/pods/global-pull-secret-syncer-qjhmr/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736853411Z namespaces/kube-system/pods/global-pull-secret-syncer-qjhmr/global-pull-secret-syncer-qjhmr.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736930236Z namespaces/kube-system/pods/global-pull-secret-syncer-qjhmr/global-pull-secret-syncer/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736946016Z namespaces/kube-system/pods/global-pull-secret-syncer-qjhmr/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736952727Z namespaces/kube-system/pods/global-pull-secret-syncer-qjhmr/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.736960917Z namespaces/kube-system/pods/global-pull-secret-syncer-qjhmr/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.737311209Z namespaces/kube-system/pods/global-pull-secret-syncer-qjhmr/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.737365962Z namespaces/kube-system/pods/global-pull-secret-syncer-qjhmr/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.737445947Z namespaces/kube-system/pods/global-pull-secret-syncer-rngnz/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.737516672Z namespaces/kube-system/pods/global-pull-secret-syncer-rngnz/global-pull-secret-syncer-rngnz.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.737598497Z namespaces/kube-system/pods/global-pull-secret-syncer-rngnz/global-pull-secret-syncer/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.737608467Z namespaces/kube-system/pods/global-pull-secret-syncer-rngnz/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.737618708Z namespaces/kube-system/pods/global-pull-secret-syncer-rngnz/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.737661051Z namespaces/kube-system/pods/global-pull-secret-syncer-rngnz/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.738034074Z namespaces/kube-system/pods/global-pull-secret-syncer-rngnz/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.73813996Z namespaces/kube-system/pods/global-pull-secret-syncer-rngnz/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.738235416Z namespaces/kube-system/pods/konnectivity-agent-4hmls/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.738280639Z namespaces/kube-system/pods/konnectivity-agent-4hmls/konnectivity-agent-4hmls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.738376045Z namespaces/kube-system/pods/konnectivity-agent-4hmls/konnectivity-agent/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.738384766Z namespaces/kube-system/pods/konnectivity-agent-4hmls/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.738406127Z namespaces/kube-system/pods/konnectivity-agent-4hmls/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.738430318Z namespaces/kube-system/pods/konnectivity-agent-4hmls/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.739546047Z namespaces/kube-system/pods/konnectivity-agent-4hmls/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.739638933Z namespaces/kube-system/pods/konnectivity-agent-4hmls/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.739713918Z namespaces/kube-system/pods/konnectivity-agent-4rtn2/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.739770941Z namespaces/kube-system/pods/konnectivity-agent-4rtn2/konnectivity-agent-4rtn2.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.739867617Z namespaces/kube-system/pods/konnectivity-agent-4rtn2/konnectivity-agent/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.739879318Z namespaces/kube-system/pods/konnectivity-agent-4rtn2/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.739883658Z namespaces/kube-system/pods/konnectivity-agent-4rtn2/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.73991942Z namespaces/kube-system/pods/konnectivity-agent-4rtn2/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.741189349Z namespaces/kube-system/pods/konnectivity-agent-4rtn2/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.741266604Z namespaces/kube-system/pods/konnectivity-agent-4rtn2/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.741309876Z namespaces/kube-system/pods/konnectivity-agent-6cg2l/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.741340898Z namespaces/kube-system/pods/konnectivity-agent-6cg2l/konnectivity-agent-6cg2l.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.741430014Z namespaces/kube-system/pods/konnectivity-agent-6cg2l/konnectivity-agent/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.741439714Z namespaces/kube-system/pods/konnectivity-agent-6cg2l/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.741447485Z namespaces/kube-system/pods/konnectivity-agent-6cg2l/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.741493148Z namespaces/kube-system/pods/konnectivity-agent-6cg2l/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.742485659Z namespaces/kube-system/pods/konnectivity-agent-6cg2l/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.742555403Z namespaces/kube-system/pods/konnectivity-agent-6cg2l/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.742613367Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-180.ec2.internal/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.742650729Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-180.ec2.internal/kube-apiserver-proxy-ip-10-0-128-180.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.742747845Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-180.ec2.internal/haproxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.742755656Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-180.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.742761906Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-180.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.742804299Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-180.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743038623Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-180.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.74314791Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-128-180.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743212414Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-132-31.ec2.internal/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743250326Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-132-31.ec2.internal/kube-apiserver-proxy-ip-10-0-132-31.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743371494Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-132-31.ec2.internal/haproxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743385645Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-132-31.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743390905Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-132-31.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743429267Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-132-31.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743729116Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-132-31.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.74380345Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-132-31.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743844243Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743883515Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.74395093Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal/haproxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.74395837Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743962471Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.743993212Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744288841Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744360615Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744393457Z namespaces/kube-system/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744432509Z namespaces/kube-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744487583Z namespaces/kube-system/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744508254Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744555787Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/csi-snapshot-controller-operator-authentication-reader.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744640642Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/network-diagnostics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744709887Z namespaces/kube-system/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.7447645Z namespaces/kube-system/rbac.authorization.k8s.io/roles/extension-apiserver-authentication-reader.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744795512Z namespaces/kube-system/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744848775Z namespaces/kube-system/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744893588Z namespaces/open-cluster-management-agent-addon/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744903778Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.74492178Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.744942051Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/cluster-proxy.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.74509709Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/work-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.745175805Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.745198927Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.745221718Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.745422901Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.745621123Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.745776423Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.746036999Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.746319886Z namespaces/open-cluster-management-c8cc9e53-dbe8-46dd-92c8-d1cbe7ed4/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.746335777Z namespaces/open-cluster-management-c8cc9e53-dbe8-46dd-92c8-d1cbe7ed4/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.746347758Z namespaces/open-cluster-management-c8cc9e53-dbe8-46dd-92c8-d1cbe7ed4/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.746373869Z namespaces/open-cluster-management-c8cc9e53-dbe8-46dd-92c8-d1cbe7ed4/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.746570852Z namespaces/open-cluster-management-c8cc9e53-dbe8-46dd-92c8-d1cbe7ed4/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.746762144Z namespaces/open-cluster-management-c8cc9e53-dbe8-46dd-92c8-d1cbe7ed4/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.746912953Z namespaces/open-cluster-management-c8cc9e53-dbe8-46dd-92c8-d1cbe7ed4/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.74735144Z namespaces/open-cluster-management-c8cc9e53-dbe8-46dd-92c8-d1cbe7ed4/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.747587395Z namespaces/openshift-apiserver-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.747631987Z namespaces/openshift-apiserver-operator/openshift-apiserver-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.747710052Z namespaces/openshift-apiserver-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.747758585Z namespaces/openshift-apiserver-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.747826659Z namespaces/openshift-apiserver-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.747871072Z namespaces/openshift-apiserver-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.747959618Z namespaces/openshift-apiserver-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748043463Z namespaces/openshift-apiserver-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748151079Z namespaces/openshift-apiserver-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748202233Z namespaces/openshift-apiserver-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748243755Z namespaces/openshift-apiserver-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748291238Z namespaces/openshift-apiserver-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748336401Z namespaces/openshift-apiserver-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748410655Z namespaces/openshift-apiserver-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748463329Z namespaces/openshift-apiserver-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748496471Z namespaces/openshift-apiserver-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748580426Z namespaces/openshift-apiserver-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.74863888Z namespaces/openshift-apiserver-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.748700773Z namespaces/openshift-apiserver-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.7496151Z namespaces/openshift-apiserver-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.74977161Z namespaces/openshift-apiserver-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.749854495Z namespaces/openshift-apiserver-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.74993842Z namespaces/openshift-apiserver-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750061168Z namespaces/openshift-apiserver-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750211777Z namespaces/openshift-apiserver-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75041949Z namespaces/openshift-apiserver-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750495644Z namespaces/openshift-apiserver-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750533697Z namespaces/openshift-apiserver-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75058693Z namespaces/openshift-apiserver-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750619782Z namespaces/openshift-apiserver-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750679566Z namespaces/openshift-apiserver-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750712988Z namespaces/openshift-apiserver-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750804143Z namespaces/openshift-apiserver-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750880478Z namespaces/openshift-apiserver-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750942722Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.750981345Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.751059559Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.751085961Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors/openshift-apiserver-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.751180277Z namespaces/openshift-apiserver-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.751199878Z namespaces/openshift-apiserver-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.751272772Z namespaces/openshift-apiserver-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.751283303Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.751334676Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.751488516Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.751673557Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.751837627Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.752072592Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.752304336Z namespaces/openshift-apiserver-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.752338638Z namespaces/openshift-apiserver-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.752383911Z namespaces/openshift-apiserver-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.752427154Z namespaces/openshift-apiserver-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75253263Z namespaces/openshift-apiserver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.752574753Z namespaces/openshift-apiserver/openshift-apiserver.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.752717612Z namespaces/openshift-apiserver/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.752758324Z namespaces/openshift-apiserver/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.752814478Z namespaces/openshift-apiserver/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75285679Z namespaces/openshift-apiserver/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.752944716Z namespaces/openshift-apiserver/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753021231Z namespaces/openshift-apiserver/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753104076Z namespaces/openshift-apiserver/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75317409Z namespaces/openshift-apiserver/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753217743Z namespaces/openshift-apiserver/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753274386Z namespaces/openshift-apiserver/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753322369Z namespaces/openshift-apiserver/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753397644Z namespaces/openshift-apiserver/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753449687Z namespaces/openshift-apiserver/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75349528Z namespaces/openshift-apiserver/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753569185Z namespaces/openshift-apiserver/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753643909Z namespaces/openshift-apiserver/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753695142Z namespaces/openshift-apiserver/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753796439Z namespaces/openshift-apiserver/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.753907486Z namespaces/openshift-apiserver/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75398419Z namespaces/openshift-apiserver/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.754072196Z namespaces/openshift-apiserver/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.754251787Z namespaces/openshift-apiserver/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.754378605Z namespaces/openshift-apiserver/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.754544445Z namespaces/openshift-apiserver/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.754600228Z namespaces/openshift-apiserver/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75463386Z namespaces/openshift-apiserver/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.754699555Z namespaces/openshift-apiserver/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.754738057Z namespaces/openshift-apiserver/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75479595Z namespaces/openshift-apiserver/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.754833863Z namespaces/openshift-apiserver/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.754917438Z namespaces/openshift-apiserver/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.754997933Z namespaces/openshift-apiserver/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.755053216Z namespaces/openshift-apiserver/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.755094929Z namespaces/openshift-apiserver/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.755174944Z namespaces/openshift-apiserver/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.755231397Z namespaces/openshift-apiserver/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75526945Z namespaces/openshift-apiserver/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75528208Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.755334714Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.755498634Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.755661674Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.755806373Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756009676Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756221229Z namespaces/openshift-apiserver/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75624411Z namespaces/openshift-apiserver/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756336416Z namespaces/openshift-apiserver/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756354497Z namespaces/openshift-apiserver/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756645885Z namespaces/openshift-authentication-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756658025Z namespaces/openshift-authentication-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756668586Z namespaces/openshift-authentication-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756716099Z namespaces/openshift-authentication-operator/monitoring.coreos.com/servicemonitors/authentication-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756771693Z namespaces/openshift-authentication-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756794974Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756831736Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.756982496Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.757198089Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.757341188Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.757555171Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.757745683Z namespaces/openshift-authentication/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.757754493Z namespaces/openshift-authentication/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.757775295Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.757853119Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.758167909Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.758355841Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.758489599Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.758687911Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.758877023Z namespaces/openshift-cloud-controller-manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.758886303Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.758891474Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.758923346Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/leases/cloud-controller-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.75899667Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.759008421Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.759075415Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.759253026Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.759463339Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.759612548Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.759831422Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760026524Z namespaces/openshift-cloud-credential-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760035925Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760040995Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760087558Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/leases/cloud-credential-operator-leader.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760186554Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760201545Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760255248Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/servicemonitors/cloud-credential-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760324682Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760333923Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760386366Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760544066Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760733228Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.760877157Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76108661Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.761321654Z namespaces/openshift-cloud-network-config-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.761374537Z namespaces/openshift-cloud-network-config-controller/openshift-cloud-network-config-controller.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.761455462Z namespaces/openshift-cloud-network-config-controller/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.761496305Z namespaces/openshift-cloud-network-config-controller/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.761588221Z namespaces/openshift-cloud-network-config-controller/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.761637744Z namespaces/openshift-cloud-network-config-controller/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76174178Z namespaces/openshift-cloud-network-config-controller/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.761845226Z namespaces/openshift-cloud-network-config-controller/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.761937002Z namespaces/openshift-cloud-network-config-controller/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.761993856Z namespaces/openshift-cloud-network-config-controller/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.7620588Z namespaces/openshift-cloud-network-config-controller/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.762114893Z namespaces/openshift-cloud-network-config-controller/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.762265152Z namespaces/openshift-cloud-network-config-controller/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.762364729Z namespaces/openshift-cloud-network-config-controller/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.762412961Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.762558421Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.762644396Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.762698349Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76271Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.762775944Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/leases/cloud-network-config-controller-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76286872Z namespaces/openshift-cloud-network-config-controller/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.762923053Z namespaces/openshift-cloud-network-config-controller/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76303837Z namespaces/openshift-cloud-network-config-controller/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.763139477Z namespaces/openshift-cloud-network-config-controller/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.763256084Z namespaces/openshift-cloud-network-config-controller/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76335742Z namespaces/openshift-cloud-network-config-controller/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.763450786Z namespaces/openshift-cloud-network-config-controller/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.763635887Z namespaces/openshift-cloud-network-config-controller/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.763786266Z namespaces/openshift-cloud-network-config-controller/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76384668Z namespaces/openshift-cloud-network-config-controller/core/serviceaccounts/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.763910634Z namespaces/openshift-cloud-network-config-controller/core/serviceaccounts/cloud-network-config-controller.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.763975818Z namespaces/openshift-cloud-network-config-controller/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764033882Z namespaces/openshift-cloud-network-config-controller/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764093856Z namespaces/openshift-cloud-network-config-controller/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764181441Z namespaces/openshift-cloud-network-config-controller/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764268856Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76432619Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764427166Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764532363Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764590086Z namespaces/openshift-cloud-network-config-controller/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76464688Z namespaces/openshift-cloud-network-config-controller/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764710934Z namespaces/openshift-cloud-network-config-controller/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764764797Z namespaces/openshift-cloud-network-config-controller/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764822381Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764829491Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.764883744Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.765038274Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.7652982Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76546506Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.765719506Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.765919918Z namespaces/openshift-cloud-network-config-controller/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.765958581Z namespaces/openshift-cloud-network-config-controller/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.766029815Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.766041636Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.766088849Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/rolebindings/cloud-network-config-controller-rb.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.766166664Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767191097Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/roles/cloud-network-config-controller.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767257851Z namespaces/openshift-cloud-network-config-controller/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767292633Z namespaces/openshift-cloud-network-config-controller/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767390809Z namespaces/openshift-cluster-csi-drivers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767453033Z namespaces/openshift-cluster-csi-drivers/openshift-cluster-csi-drivers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767502756Z namespaces/openshift-cluster-csi-drivers/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767533908Z namespaces/openshift-cluster-csi-drivers/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767612353Z namespaces/openshift-cluster-csi-drivers/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767663556Z namespaces/openshift-cluster-csi-drivers/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767866469Z namespaces/openshift-cluster-csi-drivers/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.767966695Z namespaces/openshift-cluster-csi-drivers/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768068751Z namespaces/openshift-cluster-csi-drivers/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768144716Z namespaces/openshift-cluster-csi-drivers/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768179188Z namespaces/openshift-cluster-csi-drivers/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768234472Z namespaces/openshift-cluster-csi-drivers/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768285215Z namespaces/openshift-cluster-csi-drivers/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.76836903Z namespaces/openshift-cluster-csi-drivers/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768428644Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768464296Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768548361Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768609145Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768616655Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768654558Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/ebs-csi-aws-com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768754184Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-attacher-leader-ebs-csi-aws-com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768838129Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-resizer-ebs-csi-aws-com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.768971977Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-snapshotter-leader-ebs-csi-aws-com.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.769073404Z namespaces/openshift-cluster-csi-drivers/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.769091874Z namespaces/openshift-cluster-csi-drivers/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.769238684Z namespaces/openshift-cluster-csi-drivers/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.769300627Z namespaces/openshift-cluster-csi-drivers/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.769648779Z namespaces/openshift-cluster-csi-drivers/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.769725614Z namespaces/openshift-cluster-csi-drivers/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.770337742Z namespaces/openshift-cluster-csi-drivers/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.770483161Z namespaces/openshift-cluster-csi-drivers/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.770604448Z namespaces/openshift-cluster-csi-drivers/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77063947Z namespaces/openshift-cluster-csi-drivers/core/serviceaccounts/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.770693004Z namespaces/openshift-cluster-csi-drivers/core/serviceaccounts/aws-ebs-csi-driver-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.770735776Z namespaces/openshift-cluster-csi-drivers/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77078699Z namespaces/openshift-cluster-csi-drivers/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.770828252Z namespaces/openshift-cluster-csi-drivers/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.770893776Z namespaces/openshift-cluster-csi-drivers/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.770936009Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.770983802Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771067097Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771162303Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771204605Z namespaces/openshift-cluster-csi-drivers/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771262559Z namespaces/openshift-cluster-csi-drivers/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771306562Z namespaces/openshift-cluster-csi-drivers/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771367055Z namespaces/openshift-cluster-csi-drivers/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771421569Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77143413Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771476702Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771638782Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771824224Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.771969533Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772211628Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772420011Z namespaces/openshift-cluster-csi-drivers/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772431981Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772471434Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/aws-ebs-csi-driver-node-bmmmc.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772583861Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-driver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772594001Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-driver/csi-driver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772599242Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-driver/csi-driver/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772654965Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.7727345Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772809835Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772824505Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-liveness-probe/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772832206Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.772838146Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77290154Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773001066Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77306238Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773176967Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-node-driver-registrar/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773191738Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773201129Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773238231Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773336187Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773446104Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bmmmc/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773498507Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773522779Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/aws-ebs-csi-driver-node-f7g8w.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773633566Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-driver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773643286Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-driver/csi-driver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773655317Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-driver/csi-driver/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773714311Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773831118Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773905202Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773935884Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-liveness-probe/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773945545Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.773952015Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774005029Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774098144Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774286366Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774311697Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-node-driver-registrar/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774321798Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774330969Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774386252Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774473607Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774554052Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-f7g8w/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774589955Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774647218Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/aws-ebs-csi-driver-node-lnkhf.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774728613Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-driver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774753025Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-driver/csi-driver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774758095Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-driver/csi-driver/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774802948Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774902844Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.774971498Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775112557Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-liveness-probe/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775136279Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775144499Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775208513Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775310459Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775372823Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775430747Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-node-driver-registrar/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775442117Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775446738Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77548501Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775582506Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77564592Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lnkhf/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775707034Z namespaces/openshift-cluster-csi-drivers/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775727485Z namespaces/openshift-cluster-csi-drivers/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775799989Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77581169Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775845472Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/rolebindings/aws-ebs-csi-driver-operator-rolebinding.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775915107Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.775950909Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/roles/aws-ebs-csi-driver-operator-role.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776007933Z namespaces/openshift-cluster-csi-drivers/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776052825Z namespaces/openshift-cluster-csi-drivers/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776143041Z namespaces/openshift-cluster-machine-approver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776158692Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776166582Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776203554Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/leases/cluster-machine-approver-leader.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776272479Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776281399Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776327342Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/servicemonitors/cluster-machine-approver.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776362664Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776375855Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776433529Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776590809Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776770149Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.776911458Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.777112931Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.777469083Z namespaces/openshift-cluster-node-tuning-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.777491974Z namespaces/openshift-cluster-node-tuning-operator/openshift-cluster-node-tuning-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77758677Z namespaces/openshift-cluster-node-tuning-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.777607151Z namespaces/openshift-cluster-node-tuning-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.777686776Z namespaces/openshift-cluster-node-tuning-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.777710538Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.777859997Z namespaces/openshift-cluster-node-tuning-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.777930561Z namespaces/openshift-cluster-node-tuning-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778012056Z namespaces/openshift-cluster-node-tuning-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778093631Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778177817Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets/tuned.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77839944Z namespaces/openshift-cluster-node-tuning-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778425232Z namespaces/openshift-cluster-node-tuning-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778516097Z namespaces/openshift-cluster-node-tuning-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778542599Z namespaces/openshift-cluster-node-tuning-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778618884Z namespaces/openshift-cluster-node-tuning-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778674967Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.7787237Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778802355Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778847418Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778866929Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.778917852Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/leases/node-tuning-operator-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.779000458Z namespaces/openshift-cluster-node-tuning-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.77903764Z namespaces/openshift-cluster-node-tuning-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.780142808Z namespaces/openshift-cluster-node-tuning-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78033705Z namespaces/openshift-cluster-node-tuning-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.780507751Z namespaces/openshift-cluster-node-tuning-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.780601486Z namespaces/openshift-cluster-node-tuning-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.780838031Z namespaces/openshift-cluster-node-tuning-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781041884Z namespaces/openshift-cluster-node-tuning-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781234216Z namespaces/openshift-cluster-node-tuning-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781315141Z namespaces/openshift-cluster-node-tuning-operator/core/serviceaccounts/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781353653Z namespaces/openshift-cluster-node-tuning-operator/core/serviceaccounts/cluster-node-tuning-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781439168Z namespaces/openshift-cluster-node-tuning-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781487871Z namespaces/openshift-cluster-node-tuning-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781558976Z namespaces/openshift-cluster-node-tuning-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781606319Z namespaces/openshift-cluster-node-tuning-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781692094Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781753268Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781860364Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.781959641Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.782045656Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.782067017Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.782257569Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78227378Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/prometheusrules/node-tuning-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.782392087Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.782425629Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors/node-tuning-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.782508835Z namespaces/openshift-cluster-node-tuning-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.782541286Z namespaces/openshift-cluster-node-tuning-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.782620082Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.782645843Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.782707377Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78292416Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.783357677Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78357681Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.783863968Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.784144676Z namespaces/openshift-cluster-node-tuning-operator/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78420858Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-874qv/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.784269553Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-874qv/tuned-874qv.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.784392981Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-874qv/tuned/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.784400572Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-874qv/tuned/tuned/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.784411682Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-874qv/tuned/tuned/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.784469996Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-874qv/tuned/tuned/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.784616175Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-874qv/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.784706511Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-874qv/tuned/tuned/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.784848669Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-ggsdl/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.784903923Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-ggsdl/tuned-ggsdl.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785004509Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-ggsdl/tuned/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78501359Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-ggsdl/tuned/tuned/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78501891Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-ggsdl/tuned/tuned/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785060192Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-ggsdl/tuned/tuned/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785226863Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-ggsdl/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785312438Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-ggsdl/tuned/tuned/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785364761Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-whwbj/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785521101Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-whwbj/tuned-whwbj.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785614027Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-whwbj/tuned/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785627508Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-whwbj/tuned/tuned/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785632968Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-whwbj/tuned/tuned/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78567602Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-whwbj/tuned/tuned/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785809589Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-whwbj/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785892494Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-whwbj/tuned/tuned/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.785937867Z namespaces/openshift-cluster-node-tuning-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78599018Z namespaces/openshift-cluster-node-tuning-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.786043893Z namespaces/openshift-cluster-node-tuning-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.786096366Z namespaces/openshift-cluster-node-tuning-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.786192582Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.786565866Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.786602598Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-128-180.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.786737396Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-132-31.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.786849253Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-142-99.ec2.internal.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.786904347Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/tuneds/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.786943209Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/tuneds/default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787063956Z namespaces/openshift-cluster-samples-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787182674Z namespaces/openshift-cluster-samples-operator/openshift-cluster-samples-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787272309Z namespaces/openshift-cluster-samples-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787334723Z namespaces/openshift-cluster-samples-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787398047Z namespaces/openshift-cluster-samples-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787432259Z namespaces/openshift-cluster-samples-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787515254Z namespaces/openshift-cluster-samples-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787628781Z namespaces/openshift-cluster-samples-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787725507Z namespaces/openshift-cluster-samples-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787783261Z namespaces/openshift-cluster-samples-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787824953Z namespaces/openshift-cluster-samples-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787864886Z namespaces/openshift-cluster-samples-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787905118Z namespaces/openshift-cluster-samples-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.787985833Z namespaces/openshift-cluster-samples-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.788049667Z namespaces/openshift-cluster-samples-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.788140603Z namespaces/openshift-cluster-samples-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.788229168Z namespaces/openshift-cluster-samples-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.788291632Z namespaces/openshift-cluster-samples-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.788334795Z namespaces/openshift-cluster-samples-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.788564089Z namespaces/openshift-cluster-samples-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.788652535Z namespaces/openshift-cluster-samples-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.788783413Z namespaces/openshift-cluster-samples-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.788862917Z namespaces/openshift-cluster-samples-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.788988065Z namespaces/openshift-cluster-samples-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789115783Z namespaces/openshift-cluster-samples-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789301535Z namespaces/openshift-cluster-samples-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789358868Z namespaces/openshift-cluster-samples-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78938559Z namespaces/openshift-cluster-samples-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789457214Z namespaces/openshift-cluster-samples-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789499717Z namespaces/openshift-cluster-samples-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789572441Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789609704Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789697419Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789774484Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789823837Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.78986487Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789915873Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.789958615Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/prometheusrules/samples-operator-alerts.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.790074923Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.790141327Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors/cluster-samples-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.790217611Z namespaces/openshift-cluster-samples-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.790291876Z namespaces/openshift-cluster-samples-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79035759Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79036484Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.790420794Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.790605315Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.790820839Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.790989039Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.791461828Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.791679452Z namespaces/openshift-cluster-samples-operator/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.791686662Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.791735765Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator-65db858894-7mng6.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.791871664Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator-watch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.791891815Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator-watch/cluster-samples-operator-watch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.791900885Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.791917557Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.792011082Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.792088897Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.792149551Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.792173652Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator/cluster-samples-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.792186473Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator/cluster-samples-operator/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.792235436Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator/cluster-samples-operator/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.792833773Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator/cluster-samples-operator/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.792902827Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-65db858894-7mng6/cluster-samples-operator/cluster-samples-operator/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79293867Z namespaces/openshift-cluster-samples-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.792978452Z namespaces/openshift-cluster-samples-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793050596Z namespaces/openshift-cluster-samples-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793088409Z namespaces/openshift-cluster-samples-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793200746Z namespaces/openshift-cluster-storage-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793251169Z namespaces/openshift-cluster-storage-operator/openshift-cluster-storage-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793315763Z namespaces/openshift-cluster-storage-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793357386Z namespaces/openshift-cluster-storage-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793437411Z namespaces/openshift-cluster-storage-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793480253Z namespaces/openshift-cluster-storage-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793567799Z namespaces/openshift-cluster-storage-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793670435Z namespaces/openshift-cluster-storage-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793763931Z namespaces/openshift-cluster-storage-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793826095Z namespaces/openshift-cluster-storage-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793870807Z namespaces/openshift-cluster-storage-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793927171Z namespaces/openshift-cluster-storage-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.793959263Z namespaces/openshift-cluster-storage-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.794048658Z namespaces/openshift-cluster-storage-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79408062Z namespaces/openshift-cluster-storage-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.794136684Z namespaces/openshift-cluster-storage-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.794246321Z namespaces/openshift-cluster-storage-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.794285213Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.794299244Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.794353097Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/data-source-validator-leader.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.794433222Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/snapshot-controller-leader.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.794519557Z namespaces/openshift-cluster-storage-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79456495Z namespaces/openshift-cluster-storage-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.794682577Z namespaces/openshift-cluster-storage-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.794765713Z namespaces/openshift-cluster-storage-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.795139426Z namespaces/openshift-cluster-storage-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.795249023Z namespaces/openshift-cluster-storage-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.795383501Z namespaces/openshift-cluster-storage-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.795555482Z namespaces/openshift-cluster-storage-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.795731682Z namespaces/openshift-cluster-storage-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.795804537Z namespaces/openshift-cluster-storage-operator/core/serviceaccounts/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.795888142Z namespaces/openshift-cluster-storage-operator/core/serviceaccounts/csi-snapshot-controller-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.795970567Z namespaces/openshift-cluster-storage-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79602025Z namespaces/openshift-cluster-storage-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796111646Z namespaces/openshift-cluster-storage-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796190571Z namespaces/openshift-cluster-storage-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796294517Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796326159Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796429305Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79649655Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796562304Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796601566Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79667078Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796719474Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/prometheusrules/prometheus.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796811979Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796843721Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors/cluster-storage-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796904495Z namespaces/openshift-cluster-storage-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.796946268Z namespaces/openshift-cluster-storage-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.797119738Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79715529Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.797207924Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.797367674Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.797543905Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.797689334Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.797898706Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798091228Z namespaces/openshift-cluster-storage-operator/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79811602Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78495597f6-2jz9x/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798154232Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78495597f6-2jz9x/volume-data-source-validator-78495597f6-2jz9x.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798234697Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78495597f6-2jz9x/volume-data-source-validator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798245638Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78495597f6-2jz9x/volume-data-source-validator/volume-data-source-validator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798251498Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78495597f6-2jz9x/volume-data-source-validator/volume-data-source-validator/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798269749Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78495597f6-2jz9x/volume-data-source-validator/volume-data-source-validator/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798386197Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78495597f6-2jz9x/volume-data-source-validator/volume-data-source-validator/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798476012Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78495597f6-2jz9x/volume-data-source-validator/volume-data-source-validator/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798514825Z namespaces/openshift-cluster-storage-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798573378Z namespaces/openshift-cluster-storage-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798642592Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798656213Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798683725Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/rolebindings/csi-snapshot-controller-operator-role.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79876404Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798826134Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/roles/csi-snapshot-controller-operator-role.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798914159Z namespaces/openshift-cluster-storage-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.798966233Z namespaces/openshift-cluster-storage-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799076119Z namespaces/openshift-cluster-version/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799139823Z namespaces/openshift-cluster-version/openshift-cluster-version.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799299523Z namespaces/openshift-cluster-version/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799340505Z namespaces/openshift-cluster-version/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799387059Z namespaces/openshift-cluster-version/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799436862Z namespaces/openshift-cluster-version/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799520447Z namespaces/openshift-cluster-version/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799600832Z namespaces/openshift-cluster-version/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799680606Z namespaces/openshift-cluster-version/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.79974245Z namespaces/openshift-cluster-version/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799787013Z namespaces/openshift-cluster-version/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799840976Z namespaces/openshift-cluster-version/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799885209Z namespaces/openshift-cluster-version/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.799967714Z namespaces/openshift-cluster-version/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.800014307Z namespaces/openshift-cluster-version/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.80006602Z namespaces/openshift-cluster-version/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.800200169Z namespaces/openshift-cluster-version/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.800256992Z namespaces/openshift-cluster-version/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.800264643Z namespaces/openshift-cluster-version/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.800302965Z namespaces/openshift-cluster-version/coordination.k8s.io/leases/version.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.80037962Z namespaces/openshift-cluster-version/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.800425443Z namespaces/openshift-cluster-version/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.800531209Z namespaces/openshift-cluster-version/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.800603314Z namespaces/openshift-cluster-version/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.80070691Z namespaces/openshift-cluster-version/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.800790565Z namespaces/openshift-cluster-version/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.800878701Z namespaces/openshift-cluster-version/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801004608Z namespaces/openshift-cluster-version/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801201151Z namespaces/openshift-cluster-version/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801271605Z namespaces/openshift-cluster-version/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801321318Z namespaces/openshift-cluster-version/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801377271Z namespaces/openshift-cluster-version/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801423754Z namespaces/openshift-cluster-version/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801468277Z namespaces/openshift-cluster-version/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.80150995Z namespaces/openshift-cluster-version/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801590095Z namespaces/openshift-cluster-version/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.80167152Z namespaces/openshift-cluster-version/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801716823Z namespaces/openshift-cluster-version/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801757235Z namespaces/openshift-cluster-version/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801822709Z namespaces/openshift-cluster-version/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801871442Z namespaces/openshift-cluster-version/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801915175Z namespaces/openshift-cluster-version/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801924445Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.801982309Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.802153499Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.802353962Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.802492111Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.802688713Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.802887795Z namespaces/openshift-cluster-version/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.802931148Z namespaces/openshift-cluster-version/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.802982521Z namespaces/openshift-cluster-version/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803039634Z namespaces/openshift-cluster-version/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803151721Z namespaces/openshift-config-managed/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803204235Z namespaces/openshift-config-managed/openshift-config-managed.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803269028Z namespaces/openshift-config-managed/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803316322Z namespaces/openshift-config-managed/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803380406Z namespaces/openshift-config-managed/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803418038Z namespaces/openshift-config-managed/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803510103Z namespaces/openshift-config-managed/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803588808Z namespaces/openshift-config-managed/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803669213Z namespaces/openshift-config-managed/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803819913Z namespaces/openshift-config-managed/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803867365Z namespaces/openshift-config-managed/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803915299Z namespaces/openshift-config-managed/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.803960591Z namespaces/openshift-config-managed/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.804038246Z namespaces/openshift-config-managed/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.80409389Z namespaces/openshift-config-managed/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.804193866Z namespaces/openshift-config-managed/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.804287191Z namespaces/openshift-config-managed/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.804363626Z namespaces/openshift-config-managed/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.804405029Z namespaces/openshift-config-managed/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.80766191Z namespaces/openshift-config-managed/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.807742665Z namespaces/openshift-config-managed/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.807832381Z namespaces/openshift-config-managed/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.807913076Z namespaces/openshift-config-managed/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.80799148Z namespaces/openshift-config-managed/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808174102Z namespaces/openshift-config-managed/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808284009Z namespaces/openshift-config-managed/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808353043Z namespaces/openshift-config-managed/core/configmaps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808396546Z namespaces/openshift-config-managed/core/configmaps/console-public.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808490051Z namespaces/openshift-config-managed/core/configmaps/openshift-network-features.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808556466Z namespaces/openshift-config-managed/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808601048Z namespaces/openshift-config-managed/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808639111Z namespaces/openshift-config-managed/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808701535Z namespaces/openshift-config-managed/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808762418Z namespaces/openshift-config-managed/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808809971Z namespaces/openshift-config-managed/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808894896Z namespaces/openshift-config-managed/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.808976201Z namespaces/openshift-config-managed/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.809013184Z namespaces/openshift-config-managed/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.809049746Z namespaces/openshift-config-managed/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.809137162Z namespaces/openshift-config-managed/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.809192905Z namespaces/openshift-config-managed/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.809258359Z namespaces/openshift-config-managed/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.80927181Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.809326833Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.809514105Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.809714487Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.809845105Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810051408Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810294313Z namespaces/openshift-config-managed/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810342786Z namespaces/openshift-config-managed/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810389399Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810398669Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810464733Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/rolebindings/openshift-network-public-role-binding.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810509656Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810545718Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/roles/openshift-network-public-role.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810628974Z namespaces/openshift-config-managed/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810673586Z namespaces/openshift-config-managed/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810792354Z namespaces/openshift-config-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810806775Z namespaces/openshift-config-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810813275Z namespaces/openshift-config-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810837127Z namespaces/openshift-config-operator/monitoring.coreos.com/servicemonitors/config-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810917312Z namespaces/openshift-config-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810926872Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.810953324Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.811144386Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.811329797Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.811465815Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.811668438Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.811893722Z namespaces/openshift-config/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.811943025Z namespaces/openshift-config/openshift-config.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812127566Z namespaces/openshift-config/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81218811Z namespaces/openshift-config/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812278776Z namespaces/openshift-config/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812361341Z namespaces/openshift-config/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812450747Z namespaces/openshift-config/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812532012Z namespaces/openshift-config/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812616187Z namespaces/openshift-config/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81267298Z namespaces/openshift-config/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812711952Z namespaces/openshift-config/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812780577Z namespaces/openshift-config/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812813529Z namespaces/openshift-config/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812904124Z namespaces/openshift-config/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.812950857Z namespaces/openshift-config/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81299094Z namespaces/openshift-config/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.813069765Z namespaces/openshift-config/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.813136939Z namespaces/openshift-config/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.813215424Z namespaces/openshift-config/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.813371703Z namespaces/openshift-config/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81347388Z namespaces/openshift-config/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.813559705Z namespaces/openshift-config/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.813653431Z namespaces/openshift-config/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.813722705Z namespaces/openshift-config/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.813868454Z namespaces/openshift-config/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814142071Z namespaces/openshift-config/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814248938Z namespaces/openshift-config/core/secrets/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81429201Z namespaces/openshift-config/core/secrets/pull-secret.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814340583Z namespaces/openshift-config/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814380916Z namespaces/openshift-config/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814438629Z namespaces/openshift-config/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814477352Z namespaces/openshift-config/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814538836Z namespaces/openshift-config/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814578328Z namespaces/openshift-config/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814667673Z namespaces/openshift-config/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.8147784Z namespaces/openshift-config/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814827434Z namespaces/openshift-config/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814867796Z namespaces/openshift-config/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814921879Z namespaces/openshift-config/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.814965452Z namespaces/openshift-config/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.815017765Z namespaces/openshift-config/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.815027156Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.815081909Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81526629Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.815449812Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.815593831Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.815790553Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.815977125Z namespaces/openshift-config/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816017597Z namespaces/openshift-config/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81606347Z namespaces/openshift-config/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816138015Z namespaces/openshift-config/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81623553Z namespaces/openshift-console-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816266983Z namespaces/openshift-console-operator/openshift-console-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81639478Z namespaces/openshift-console-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816434493Z namespaces/openshift-console-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816513438Z namespaces/openshift-console-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816543549Z namespaces/openshift-console-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816633245Z namespaces/openshift-console-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816759343Z namespaces/openshift-console-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816886211Z namespaces/openshift-console-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816910302Z namespaces/openshift-console-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816958295Z namespaces/openshift-console-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.816994848Z namespaces/openshift-console-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81703936Z namespaces/openshift-console-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.817150127Z namespaces/openshift-console-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.81720413Z namespaces/openshift-console-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.817259854Z namespaces/openshift-console-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.817386522Z namespaces/openshift-console-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.817427184Z namespaces/openshift-console-operator/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.817437715Z namespaces/openshift-console-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.817472527Z namespaces/openshift-console-operator/coordination.k8s.io/leases/console-operator-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.817539501Z namespaces/openshift-console-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.817593405Z namespaces/openshift-console-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.818190161Z namespaces/openshift-console-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.818294958Z namespaces/openshift-console-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.818625708Z namespaces/openshift-console-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.818745946Z namespaces/openshift-console-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.818877174Z namespaces/openshift-console-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819003812Z namespaces/openshift-console-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819200444Z namespaces/openshift-console-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819308191Z namespaces/openshift-console-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819376705Z namespaces/openshift-console-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819444859Z namespaces/openshift-console-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819478311Z namespaces/openshift-console-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819552006Z namespaces/openshift-console-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819574557Z namespaces/openshift-console-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819691264Z namespaces/openshift-console-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819757248Z namespaces/openshift-console-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819831523Z namespaces/openshift-console-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819850614Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819911738Z namespaces/openshift-console-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.819985122Z namespaces/openshift-console-operator/monitoring.coreos.com/prometheusrules/cluster-monitoring-prometheus-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.820049596Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.820097189Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors/console-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.820166304Z namespaces/openshift-console-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.820226587Z namespaces/openshift-console-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.82026679Z namespaces/openshift-console-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.820276531Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.820338894Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.820483153Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.820662424Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.820801803Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.821004816Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.821223259Z namespaces/openshift-console-operator/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.82123573Z namespaces/openshift-console-operator/pods/console-operator-7866497494-nkbnv/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.821264421Z namespaces/openshift-console-operator/pods/console-operator-7866497494-nkbnv/console-operator-7866497494-nkbnv.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.821459824Z namespaces/openshift-console-operator/pods/console-operator-7866497494-nkbnv/console-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.821477205Z namespaces/openshift-console-operator/pods/console-operator-7866497494-nkbnv/console-operator/console-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.821484805Z namespaces/openshift-console-operator/pods/console-operator-7866497494-nkbnv/console-operator/console-operator/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.821512057Z namespaces/openshift-console-operator/pods/console-operator-7866497494-nkbnv/console-operator/console-operator/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822070721Z namespaces/openshift-console-operator/pods/console-operator-7866497494-nkbnv/console-operator/console-operator/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822138006Z namespaces/openshift-console-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822247382Z namespaces/openshift-console-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822328747Z namespaces/openshift-console-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822434824Z namespaces/openshift-console-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822493008Z namespaces/openshift-console-user-settings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822503928Z namespaces/openshift-console-user-settings/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822511969Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822572862Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822718002Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.822894572Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.823035151Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.823309238Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.823537202Z namespaces/openshift-console/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.823559154Z namespaces/openshift-console/openshift-console.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.823872933Z namespaces/openshift-console/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.823915745Z namespaces/openshift-console/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.82398712Z namespaces/openshift-console/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.824024572Z namespaces/openshift-console/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.82414592Z namespaces/openshift-console/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.824323201Z namespaces/openshift-console/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.824607398Z namespaces/openshift-console/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.824646751Z namespaces/openshift-console/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.824674983Z namespaces/openshift-console/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.824753957Z namespaces/openshift-console/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.82479332Z namespaces/openshift-console/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.824890936Z namespaces/openshift-console/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.824946949Z namespaces/openshift-console/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.824998963Z namespaces/openshift-console/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.825070987Z namespaces/openshift-console/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.825150092Z namespaces/openshift-console/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.825201605Z namespaces/openshift-console/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.825803352Z namespaces/openshift-console/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.825889788Z namespaces/openshift-console/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.826294463Z namespaces/openshift-console/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.826371267Z namespaces/openshift-console/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.826559049Z namespaces/openshift-console/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.826684597Z namespaces/openshift-console/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.826882249Z namespaces/openshift-console/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.826942193Z namespaces/openshift-console/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.826992456Z namespaces/openshift-console/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.82706028Z namespaces/openshift-console/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827113943Z namespaces/openshift-console/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827209539Z namespaces/openshift-console/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827243111Z namespaces/openshift-console/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827329537Z namespaces/openshift-console/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827405912Z namespaces/openshift-console/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827464785Z namespaces/openshift-console/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827502737Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.82755237Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827589503Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors/console.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827658067Z namespaces/openshift-console/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827696539Z namespaces/openshift-console/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827766814Z namespaces/openshift-console/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827774304Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827825627Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.827977177Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.828199041Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.828341179Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.828555452Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.828768466Z namespaces/openshift-console/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.828787547Z namespaces/openshift-console/pods/console-6466cc997d-hbh2z/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.82883515Z namespaces/openshift-console/pods/console-6466cc997d-hbh2z/console-6466cc997d-hbh2z.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.828950527Z namespaces/openshift-console/pods/console-6466cc997d-hbh2z/console/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.828965528Z namespaces/openshift-console/pods/console-6466cc997d-hbh2z/console/console/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.828984039Z namespaces/openshift-console/pods/console-6466cc997d-hbh2z/console/console/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.82899703Z namespaces/openshift-console/pods/console-6466cc997d-hbh2z/console/console/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.829139009Z namespaces/openshift-console/pods/console-6466cc997d-hbh2z/console/console/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.829235394Z namespaces/openshift-console/pods/console-6466cc997d-hbh2z/console/console/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.829271927Z namespaces/openshift-console/pods/downloads-68c9cbb497-7jpdn/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.829330071Z namespaces/openshift-console/pods/downloads-68c9cbb497-7jpdn/downloads-68c9cbb497-7jpdn.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.829439067Z namespaces/openshift-console/pods/downloads-68c9cbb497-7jpdn/download-server/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.829446988Z namespaces/openshift-console/pods/downloads-68c9cbb497-7jpdn/download-server/download-server/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.829451278Z namespaces/openshift-console/pods/downloads-68c9cbb497-7jpdn/download-server/download-server/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.82948536Z namespaces/openshift-console/pods/downloads-68c9cbb497-7jpdn/download-server/download-server/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.829853763Z namespaces/openshift-console/pods/downloads-68c9cbb497-7jpdn/download-server/download-server/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.829943029Z namespaces/openshift-console/pods/downloads-68c9cbb497-7jpdn/download-server/download-server/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.829995552Z namespaces/openshift-console/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830036614Z namespaces/openshift-console/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830110739Z namespaces/openshift-console/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.8302954Z namespaces/openshift-console/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830433869Z namespaces/openshift-controller-manager-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830468651Z namespaces/openshift-controller-manager-operator/openshift-controller-manager-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830546546Z namespaces/openshift-controller-manager-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830580528Z namespaces/openshift-controller-manager-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830630921Z namespaces/openshift-controller-manager-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830663783Z namespaces/openshift-controller-manager-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830791501Z namespaces/openshift-controller-manager-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830873376Z namespaces/openshift-controller-manager-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.830982593Z namespaces/openshift-controller-manager-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831033776Z namespaces/openshift-controller-manager-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831066678Z namespaces/openshift-controller-manager-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831135592Z namespaces/openshift-controller-manager-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831218317Z namespaces/openshift-controller-manager-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831305363Z namespaces/openshift-controller-manager-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831362696Z namespaces/openshift-controller-manager-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831395558Z namespaces/openshift-controller-manager-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831482253Z namespaces/openshift-controller-manager-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831559248Z namespaces/openshift-controller-manager-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831604241Z namespaces/openshift-controller-manager-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831759621Z namespaces/openshift-controller-manager-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831865567Z namespaces/openshift-controller-manager-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.831949233Z namespaces/openshift-controller-manager-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832033698Z namespaces/openshift-controller-manager-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832184997Z namespaces/openshift-controller-manager-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832330526Z namespaces/openshift-controller-manager-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832517858Z namespaces/openshift-controller-manager-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.83254781Z namespaces/openshift-controller-manager-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832609173Z namespaces/openshift-controller-manager-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832665407Z namespaces/openshift-controller-manager-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832700099Z namespaces/openshift-controller-manager-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832782774Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832825577Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832919113Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.832995587Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.83304438Z namespaces/openshift-controller-manager-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.833082932Z namespaces/openshift-controller-manager-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.833153367Z namespaces/openshift-controller-manager-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.833182369Z namespaces/openshift-controller-manager-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.833248623Z namespaces/openshift-controller-manager-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.833256183Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.833301366Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.833454666Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.833637957Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.833777246Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.833983048Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834194011Z namespaces/openshift-controller-manager-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834237164Z namespaces/openshift-controller-manager-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834299428Z namespaces/openshift-controller-manager-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834315489Z namespaces/openshift-controller-manager-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834414855Z namespaces/openshift-controller-manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834464658Z namespaces/openshift-controller-manager/openshift-controller-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834520012Z namespaces/openshift-controller-manager/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834552603Z namespaces/openshift-controller-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834795089Z namespaces/openshift-controller-manager/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834839321Z namespaces/openshift-controller-manager/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.834928467Z namespaces/openshift-controller-manager/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835007102Z namespaces/openshift-controller-manager/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835093937Z namespaces/openshift-controller-manager/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835175802Z namespaces/openshift-controller-manager/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835226785Z namespaces/openshift-controller-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835290729Z namespaces/openshift-controller-manager/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835329802Z namespaces/openshift-controller-manager/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835415327Z namespaces/openshift-controller-manager/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835455429Z namespaces/openshift-controller-manager/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835500512Z namespaces/openshift-controller-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835587038Z namespaces/openshift-controller-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835640341Z namespaces/openshift-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835721746Z namespaces/openshift-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835772319Z namespaces/openshift-controller-manager/coordination.k8s.io/leases/openshift-master-controllers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835861284Z namespaces/openshift-controller-manager/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.835888346Z namespaces/openshift-controller-manager/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836008133Z namespaces/openshift-controller-manager/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836086648Z namespaces/openshift-controller-manager/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836219557Z namespaces/openshift-controller-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836298162Z namespaces/openshift-controller-manager/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836379576Z namespaces/openshift-controller-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836511845Z namespaces/openshift-controller-manager/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836678035Z namespaces/openshift-controller-manager/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836745409Z namespaces/openshift-controller-manager/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836787742Z namespaces/openshift-controller-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836844355Z namespaces/openshift-controller-manager/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836881667Z namespaces/openshift-controller-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836932891Z namespaces/openshift-controller-manager/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.836986024Z namespaces/openshift-controller-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.837073679Z namespaces/openshift-controller-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.837184226Z namespaces/openshift-controller-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.837353027Z namespaces/openshift-controller-manager/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.83739793Z namespaces/openshift-controller-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.837431661Z namespaces/openshift-controller-manager/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.837483255Z namespaces/openshift-controller-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.837538148Z namespaces/openshift-controller-manager/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.837545248Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.837558749Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.837807805Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.837996967Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.838142725Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.838386721Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.838571422Z namespaces/openshift-controller-manager/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.838604164Z namespaces/openshift-controller-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.838658988Z namespaces/openshift-controller-manager/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.838688369Z namespaces/openshift-controller-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.838823158Z namespaces/openshift-dns-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.83886484Z namespaces/openshift-dns-operator/openshift-dns-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.838935454Z namespaces/openshift-dns-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.838962476Z namespaces/openshift-dns-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839035631Z namespaces/openshift-dns-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839064533Z namespaces/openshift-dns-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.83918819Z namespaces/openshift-dns-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839272535Z namespaces/openshift-dns-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839355711Z namespaces/openshift-dns-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839415634Z namespaces/openshift-dns-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839463327Z namespaces/openshift-dns-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.83950889Z namespaces/openshift-dns-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839549272Z namespaces/openshift-dns-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839632858Z namespaces/openshift-dns-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839693192Z namespaces/openshift-dns-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839730604Z namespaces/openshift-dns-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839815699Z namespaces/openshift-dns-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839888593Z namespaces/openshift-dns-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.839932306Z namespaces/openshift-dns-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.840036003Z namespaces/openshift-dns-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.840145569Z namespaces/openshift-dns-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.840249156Z namespaces/openshift-dns-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.840332761Z namespaces/openshift-dns-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.840420186Z namespaces/openshift-dns-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.840541814Z namespaces/openshift-dns-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.840725215Z namespaces/openshift-dns-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.840860944Z namespaces/openshift-dns-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.840896526Z namespaces/openshift-dns-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.8409608Z namespaces/openshift-dns-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.840996872Z namespaces/openshift-dns-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841048465Z namespaces/openshift-dns-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841078867Z namespaces/openshift-dns-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841211955Z namespaces/openshift-dns-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841297231Z namespaces/openshift-dns-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841345194Z namespaces/openshift-dns-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841382746Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841438659Z namespaces/openshift-dns-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841467821Z namespaces/openshift-dns-operator/monitoring.coreos.com/prometheusrules/dns.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841543026Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841573928Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors/dns-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841633821Z namespaces/openshift-dns-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841661643Z namespaces/openshift-dns-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841726077Z namespaces/openshift-dns-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841737698Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841753579Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.841921039Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.842118122Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.842325954Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.842661745Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.842861907Z namespaces/openshift-dns-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.84289509Z namespaces/openshift-dns-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.842967914Z namespaces/openshift-dns-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.843000296Z namespaces/openshift-dns-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.843104602Z namespaces/openshift-dns/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.843180917Z namespaces/openshift-dns/openshift-dns.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.843252672Z namespaces/openshift-dns/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.843283863Z namespaces/openshift-dns/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844013849Z namespaces/openshift-dns/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844049361Z namespaces/openshift-dns/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844249353Z namespaces/openshift-dns/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844329078Z namespaces/openshift-dns/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844409443Z namespaces/openshift-dns/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844455376Z namespaces/openshift-dns/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844495609Z namespaces/openshift-dns/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844551322Z namespaces/openshift-dns/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844604255Z namespaces/openshift-dns/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844693861Z namespaces/openshift-dns/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844749834Z namespaces/openshift-dns/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844782826Z namespaces/openshift-dns/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844882932Z namespaces/openshift-dns/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844941646Z namespaces/openshift-dns/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.844978648Z namespaces/openshift-dns/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.845088685Z namespaces/openshift-dns/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.845204582Z namespaces/openshift-dns/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.845436727Z namespaces/openshift-dns/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.845517082Z namespaces/openshift-dns/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.845781888Z namespaces/openshift-dns/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.845901965Z namespaces/openshift-dns/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846084237Z namespaces/openshift-dns/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846198644Z namespaces/openshift-dns/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846255917Z namespaces/openshift-dns/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846333482Z namespaces/openshift-dns/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846385755Z namespaces/openshift-dns/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846453999Z namespaces/openshift-dns/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846476681Z namespaces/openshift-dns/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846571107Z namespaces/openshift-dns/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846654542Z namespaces/openshift-dns/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846720646Z namespaces/openshift-dns/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846755588Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846798441Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846832203Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors/dns-default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846894897Z namespaces/openshift-dns/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846924349Z namespaces/openshift-dns/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846982902Z namespaces/openshift-dns/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.846989963Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.847033955Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.847193385Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.847373347Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.847507685Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.847704337Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.847956723Z namespaces/openshift-dns/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.847969543Z namespaces/openshift-dns/pods/dns-default-6j895/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848011996Z namespaces/openshift-dns/pods/dns-default-6j895/dns-default-6j895.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848117593Z namespaces/openshift-dns/pods/dns-default-6j895/dns/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848140654Z namespaces/openshift-dns/pods/dns-default-6j895/dns/dns/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848155175Z namespaces/openshift-dns/pods/dns-default-6j895/dns/dns/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848191317Z namespaces/openshift-dns/pods/dns-default-6j895/dns/dns/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848289253Z namespaces/openshift-dns/pods/dns-default-6j895/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848362058Z namespaces/openshift-dns/pods/dns-default-6j895/dns/dns/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.84839601Z namespaces/openshift-dns/pods/dns-default-6j895/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.84840539Z namespaces/openshift-dns/pods/dns-default-6j895/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848416431Z namespaces/openshift-dns/pods/dns-default-6j895/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848467194Z namespaces/openshift-dns/pods/dns-default-6j895/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848688258Z namespaces/openshift-dns/pods/dns-default-6j895/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848754352Z namespaces/openshift-dns/pods/dns-default-6j895/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848790944Z namespaces/openshift-dns/pods/dns-default-j6rxh/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848829806Z namespaces/openshift-dns/pods/dns-default-j6rxh/dns-default-j6rxh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848913022Z namespaces/openshift-dns/pods/dns-default-j6rxh/dns/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848922662Z namespaces/openshift-dns/pods/dns-default-j6rxh/dns/dns/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848934753Z namespaces/openshift-dns/pods/dns-default-j6rxh/dns/dns/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.848969155Z namespaces/openshift-dns/pods/dns-default-j6rxh/dns/dns/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849072692Z namespaces/openshift-dns/pods/dns-default-j6rxh/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849143526Z namespaces/openshift-dns/pods/dns-default-j6rxh/dns/dns/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.84921753Z namespaces/openshift-dns/pods/dns-default-j6rxh/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849227251Z namespaces/openshift-dns/pods/dns-default-j6rxh/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849231731Z namespaces/openshift-dns/pods/dns-default-j6rxh/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849279414Z namespaces/openshift-dns/pods/dns-default-j6rxh/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849378451Z namespaces/openshift-dns/pods/dns-default-j6rxh/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849445175Z namespaces/openshift-dns/pods/dns-default-j6rxh/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849504328Z namespaces/openshift-dns/pods/dns-default-qtjkf/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849544081Z namespaces/openshift-dns/pods/dns-default-qtjkf/dns-default-qtjkf.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849625926Z namespaces/openshift-dns/pods/dns-default-qtjkf/dns/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849634286Z namespaces/openshift-dns/pods/dns-default-qtjkf/dns/dns/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849645457Z namespaces/openshift-dns/pods/dns-default-qtjkf/dns/dns/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.84970046Z namespaces/openshift-dns/pods/dns-default-qtjkf/dns/dns/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849832399Z namespaces/openshift-dns/pods/dns-default-qtjkf/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849902463Z namespaces/openshift-dns/pods/dns-default-qtjkf/dns/dns/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849920014Z namespaces/openshift-dns/pods/dns-default-qtjkf/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849926894Z namespaces/openshift-dns/pods/dns-default-qtjkf/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.849940925Z namespaces/openshift-dns/pods/dns-default-qtjkf/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850005939Z namespaces/openshift-dns/pods/dns-default-qtjkf/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850106236Z namespaces/openshift-dns/pods/dns-default-qtjkf/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850228963Z namespaces/openshift-dns/pods/dns-default-qtjkf/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850258325Z namespaces/openshift-dns/pods/node-resolver-254d8/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850291707Z namespaces/openshift-dns/pods/node-resolver-254d8/node-resolver-254d8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850383223Z namespaces/openshift-dns/pods/node-resolver-254d8/dns-node-resolver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850390563Z namespaces/openshift-dns/pods/node-resolver-254d8/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850412764Z namespaces/openshift-dns/pods/node-resolver-254d8/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850446837Z namespaces/openshift-dns/pods/node-resolver-254d8/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850540792Z namespaces/openshift-dns/pods/node-resolver-254d8/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850613717Z namespaces/openshift-dns/pods/node-resolver-254d8/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850676881Z namespaces/openshift-dns/pods/node-resolver-7xkbj/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850705662Z namespaces/openshift-dns/pods/node-resolver-7xkbj/node-resolver-7xkbj.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850807779Z namespaces/openshift-dns/pods/node-resolver-7xkbj/dns-node-resolver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850817029Z namespaces/openshift-dns/pods/node-resolver-7xkbj/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85082366Z namespaces/openshift-dns/pods/node-resolver-7xkbj/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850854082Z namespaces/openshift-dns/pods/node-resolver-7xkbj/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.850938567Z namespaces/openshift-dns/pods/node-resolver-7xkbj/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851018602Z namespaces/openshift-dns/pods/node-resolver-7xkbj/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851031963Z namespaces/openshift-dns/pods/node-resolver-nfsf7/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851083506Z namespaces/openshift-dns/pods/node-resolver-nfsf7/node-resolver-nfsf7.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851189203Z namespaces/openshift-dns/pods/node-resolver-nfsf7/dns-node-resolver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851204123Z namespaces/openshift-dns/pods/node-resolver-nfsf7/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851210364Z namespaces/openshift-dns/pods/node-resolver-nfsf7/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851233375Z namespaces/openshift-dns/pods/node-resolver-nfsf7/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85131742Z namespaces/openshift-dns/pods/node-resolver-nfsf7/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851381654Z namespaces/openshift-dns/pods/node-resolver-nfsf7/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851431607Z namespaces/openshift-dns/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851447919Z namespaces/openshift-dns/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851514303Z namespaces/openshift-dns/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851558105Z namespaces/openshift-dns/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851647681Z namespaces/openshift-etcd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851693274Z namespaces/openshift-etcd/openshift-etcd.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851746197Z namespaces/openshift-etcd/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85179108Z namespaces/openshift-etcd/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851851213Z namespaces/openshift-etcd/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851880945Z namespaces/openshift-etcd/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.851973091Z namespaces/openshift-etcd/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852052406Z namespaces/openshift-etcd/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852177564Z namespaces/openshift-etcd/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852237637Z namespaces/openshift-etcd/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852253508Z namespaces/openshift-etcd/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852321053Z namespaces/openshift-etcd/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852389647Z namespaces/openshift-etcd/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852511454Z namespaces/openshift-etcd/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852534676Z namespaces/openshift-etcd/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852590619Z namespaces/openshift-etcd/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852698396Z namespaces/openshift-etcd/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852784181Z namespaces/openshift-etcd/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852842165Z namespaces/openshift-etcd/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.852968202Z namespaces/openshift-etcd/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.853066448Z namespaces/openshift-etcd/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.853204147Z namespaces/openshift-etcd/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.853345746Z namespaces/openshift-etcd/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.853446992Z namespaces/openshift-etcd/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85357465Z namespaces/openshift-etcd/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85373261Z namespaces/openshift-etcd/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.853795434Z namespaces/openshift-etcd/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.853841387Z namespaces/openshift-etcd/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.853887989Z namespaces/openshift-etcd/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.853913501Z namespaces/openshift-etcd/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.853972105Z namespaces/openshift-etcd/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.853992786Z namespaces/openshift-etcd/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.854086842Z namespaces/openshift-etcd/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.854277483Z namespaces/openshift-etcd/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.854342738Z namespaces/openshift-etcd/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.854403121Z namespaces/openshift-etcd/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.854451094Z namespaces/openshift-etcd/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.854493947Z namespaces/openshift-etcd/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85453966Z namespaces/openshift-etcd/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85455396Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.854600974Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.854773494Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.854952345Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.855090704Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.855322938Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85551206Z namespaces/openshift-etcd/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.855547552Z namespaces/openshift-etcd/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.855617456Z namespaces/openshift-etcd/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.855654059Z namespaces/openshift-etcd/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.855757035Z namespaces/openshift-host-network/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.855791007Z namespaces/openshift-host-network/openshift-host-network.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.855866972Z namespaces/openshift-host-network/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.855883523Z namespaces/openshift-host-network/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.855967828Z namespaces/openshift-host-network/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856014601Z namespaces/openshift-host-network/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856096926Z namespaces/openshift-host-network/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856231604Z namespaces/openshift-host-network/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856307019Z namespaces/openshift-host-network/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856359582Z namespaces/openshift-host-network/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856401565Z namespaces/openshift-host-network/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856473849Z namespaces/openshift-host-network/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856497221Z namespaces/openshift-host-network/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856589597Z namespaces/openshift-host-network/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85664397Z namespaces/openshift-host-network/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856676932Z namespaces/openshift-host-network/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856764627Z namespaces/openshift-host-network/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856838252Z namespaces/openshift-host-network/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.856881095Z namespaces/openshift-host-network/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857001612Z namespaces/openshift-host-network/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857082677Z namespaces/openshift-host-network/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857231716Z namespaces/openshift-host-network/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857320692Z namespaces/openshift-host-network/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857430198Z namespaces/openshift-host-network/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857572497Z namespaces/openshift-host-network/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857677874Z namespaces/openshift-host-network/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857721616Z namespaces/openshift-host-network/core/resourcequotas/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85777392Z namespaces/openshift-host-network/core/resourcequotas/host-network-namespace-quotas.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857821133Z namespaces/openshift-host-network/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857872876Z namespaces/openshift-host-network/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857906968Z namespaces/openshift-host-network/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.857954711Z namespaces/openshift-host-network/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858021475Z namespaces/openshift-host-network/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858070688Z namespaces/openshift-host-network/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858189445Z namespaces/openshift-host-network/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858293192Z namespaces/openshift-host-network/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858347395Z namespaces/openshift-host-network/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858399778Z namespaces/openshift-host-network/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858415679Z namespaces/openshift-host-network/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858466862Z namespaces/openshift-host-network/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858524196Z namespaces/openshift-host-network/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858540077Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85858838Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.85874804Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.858927561Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859079791Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859310405Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859503617Z namespaces/openshift-host-network/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859546039Z namespaces/openshift-host-network/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859578671Z namespaces/openshift-host-network/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859637415Z namespaces/openshift-host-network/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859708199Z namespaces/openshift-image-registry/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859767453Z namespaces/openshift-image-registry/openshift-image-registry.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859827677Z namespaces/openshift-image-registry/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.8598774Z namespaces/openshift-image-registry/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859917072Z namespaces/openshift-image-registry/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.859959785Z namespaces/openshift-image-registry/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.860092793Z namespaces/openshift-image-registry/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.860290295Z namespaces/openshift-image-registry/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.860490138Z namespaces/openshift-image-registry/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.86053353Z namespaces/openshift-image-registry/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.860564112Z namespaces/openshift-image-registry/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.860623686Z namespaces/openshift-image-registry/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.86069106Z namespaces/openshift-image-registry/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.860836219Z namespaces/openshift-image-registry/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.860901413Z namespaces/openshift-image-registry/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.860948936Z namespaces/openshift-image-registry/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.861029931Z namespaces/openshift-image-registry/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.861075354Z namespaces/openshift-image-registry/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.861084295Z namespaces/openshift-image-registry/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.861111636Z namespaces/openshift-image-registry/coordination.k8s.io/leases/openshift-master-controllers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.861417485Z namespaces/openshift-image-registry/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.861466988Z namespaces/openshift-image-registry/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.862419367Z namespaces/openshift-image-registry/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.862530724Z namespaces/openshift-image-registry/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.862756638Z namespaces/openshift-image-registry/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.862839733Z namespaces/openshift-image-registry/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863048956Z namespaces/openshift-image-registry/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863221167Z namespaces/openshift-image-registry/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863467272Z namespaces/openshift-image-registry/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863520015Z namespaces/openshift-image-registry/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863567568Z namespaces/openshift-image-registry/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863608551Z namespaces/openshift-image-registry/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863643043Z namespaces/openshift-image-registry/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863691576Z namespaces/openshift-image-registry/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863726758Z namespaces/openshift-image-registry/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863820184Z namespaces/openshift-image-registry/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863900569Z namespaces/openshift-image-registry/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.863972083Z namespaces/openshift-image-registry/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864013336Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.86408555Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864173355Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/image-registry-operator-alerts.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864292663Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/image-registry-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864378108Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/imagestreams-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864438492Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864551489Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors/image-registry.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864593861Z namespaces/openshift-image-registry/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864636644Z namespaces/openshift-image-registry/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864679197Z namespaces/openshift-image-registry/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864701218Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864761252Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.864924632Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.865118164Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.865311436Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.865657787Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.865905863Z namespaces/openshift-image-registry/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.865915883Z namespaces/openshift-image-registry/pods/image-registry-66695db459-q5zrn/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.865936044Z namespaces/openshift-image-registry/pods/image-registry-66695db459-q5zrn/image-registry-66695db459-q5zrn.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.866065712Z namespaces/openshift-image-registry/pods/image-registry-66695db459-q5zrn/registry/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.866074533Z namespaces/openshift-image-registry/pods/image-registry-66695db459-q5zrn/registry/registry/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.866080024Z namespaces/openshift-image-registry/pods/image-registry-66695db459-q5zrn/registry/registry/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.866152998Z namespaces/openshift-image-registry/pods/image-registry-66695db459-q5zrn/registry/registry/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.867871774Z namespaces/openshift-image-registry/pods/image-registry-66695db459-q5zrn/registry/registry/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.867946339Z namespaces/openshift-image-registry/pods/image-registry-66695db459-q5zrn/registry/registry/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.86796517Z namespaces/openshift-image-registry/pods/node-ca-f4wxn/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868027774Z namespaces/openshift-image-registry/pods/node-ca-f4wxn/node-ca-f4wxn.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868097678Z namespaces/openshift-image-registry/pods/node-ca-f4wxn/node-ca/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868115899Z namespaces/openshift-image-registry/pods/node-ca-f4wxn/node-ca/node-ca/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.86812078Z namespaces/openshift-image-registry/pods/node-ca-f4wxn/node-ca/node-ca/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868187784Z namespaces/openshift-image-registry/pods/node-ca-f4wxn/node-ca/node-ca/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868356524Z namespaces/openshift-image-registry/pods/node-ca-f4wxn/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868427169Z namespaces/openshift-image-registry/pods/node-ca-f4wxn/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868469381Z namespaces/openshift-image-registry/pods/node-ca-fq5kb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868505383Z namespaces/openshift-image-registry/pods/node-ca-fq5kb/node-ca-fq5kb.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868585848Z namespaces/openshift-image-registry/pods/node-ca-fq5kb/node-ca/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868593439Z namespaces/openshift-image-registry/pods/node-ca-fq5kb/node-ca/node-ca/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868600439Z namespaces/openshift-image-registry/pods/node-ca-fq5kb/node-ca/node-ca/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868653963Z namespaces/openshift-image-registry/pods/node-ca-fq5kb/node-ca/node-ca/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868826073Z namespaces/openshift-image-registry/pods/node-ca-fq5kb/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.868898168Z namespaces/openshift-image-registry/pods/node-ca-fq5kb/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.86892918Z namespaces/openshift-image-registry/pods/node-ca-wxrdp/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869075429Z namespaces/openshift-image-registry/pods/node-ca-wxrdp/node-ca-wxrdp.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869149803Z namespaces/openshift-image-registry/pods/node-ca-wxrdp/node-ca/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869170095Z namespaces/openshift-image-registry/pods/node-ca-wxrdp/node-ca/node-ca/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869181175Z namespaces/openshift-image-registry/pods/node-ca-wxrdp/node-ca/node-ca/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869243609Z namespaces/openshift-image-registry/pods/node-ca-wxrdp/node-ca/node-ca/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.8694245Z namespaces/openshift-image-registry/pods/node-ca-wxrdp/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869490584Z namespaces/openshift-image-registry/pods/node-ca-wxrdp/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869516146Z namespaces/openshift-image-registry/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869568499Z namespaces/openshift-image-registry/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869632353Z namespaces/openshift-image-registry/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869669346Z namespaces/openshift-image-registry/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869718238Z namespaces/openshift-infra/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.86973795Z namespaces/openshift-infra/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.86974258Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869749631Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.869922571Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.870155525Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.870369479Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.870563091Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.870781614Z namespaces/openshift-ingress-canary/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.870822337Z namespaces/openshift-ingress-canary/openshift-ingress-canary.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.870896181Z namespaces/openshift-ingress-canary/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.870935874Z namespaces/openshift-ingress-canary/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871007538Z namespaces/openshift-ingress-canary/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871026609Z namespaces/openshift-ingress-canary/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871184509Z namespaces/openshift-ingress-canary/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871285165Z namespaces/openshift-ingress-canary/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871367691Z namespaces/openshift-ingress-canary/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871424834Z namespaces/openshift-ingress-canary/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871478827Z namespaces/openshift-ingress-canary/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871542191Z namespaces/openshift-ingress-canary/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871598135Z namespaces/openshift-ingress-canary/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87168173Z namespaces/openshift-ingress-canary/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871701811Z namespaces/openshift-ingress-canary/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871787986Z namespaces/openshift-ingress-canary/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871861851Z namespaces/openshift-ingress-canary/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871941306Z namespaces/openshift-ingress-canary/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.871987439Z namespaces/openshift-ingress-canary/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.872153839Z namespaces/openshift-ingress-canary/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.872274787Z namespaces/openshift-ingress-canary/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.872423456Z namespaces/openshift-ingress-canary/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.872503341Z namespaces/openshift-ingress-canary/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.872669191Z namespaces/openshift-ingress-canary/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.872793979Z namespaces/openshift-ingress-canary/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.872958499Z namespaces/openshift-ingress-canary/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873014312Z namespaces/openshift-ingress-canary/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873060755Z namespaces/openshift-ingress-canary/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873117809Z namespaces/openshift-ingress-canary/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873174332Z namespaces/openshift-ingress-canary/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873260888Z namespaces/openshift-ingress-canary/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87330184Z namespaces/openshift-ingress-canary/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873388936Z namespaces/openshift-ingress-canary/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873471041Z namespaces/openshift-ingress-canary/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873515833Z namespaces/openshift-ingress-canary/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873558106Z namespaces/openshift-ingress-canary/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873600478Z namespaces/openshift-ingress-canary/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873645361Z namespaces/openshift-ingress-canary/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873710755Z namespaces/openshift-ingress-canary/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873725436Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873767469Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.873921098Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87410199Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87426888Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.874473012Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.874737059Z namespaces/openshift-ingress-canary/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.8747477Z namespaces/openshift-ingress-canary/pods/ingress-canary-5blf5/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.874773261Z namespaces/openshift-ingress-canary/pods/ingress-canary-5blf5/ingress-canary-5blf5.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.874872707Z namespaces/openshift-ingress-canary/pods/ingress-canary-5blf5/serve-healthcheck-canary/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.874883998Z namespaces/openshift-ingress-canary/pods/ingress-canary-5blf5/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.874894138Z namespaces/openshift-ingress-canary/pods/ingress-canary-5blf5/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.874932451Z namespaces/openshift-ingress-canary/pods/ingress-canary-5blf5/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875034147Z namespaces/openshift-ingress-canary/pods/ingress-canary-5blf5/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875143514Z namespaces/openshift-ingress-canary/pods/ingress-canary-5blf5/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87524213Z namespaces/openshift-ingress-canary/pods/ingress-canary-dmmld/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875296014Z namespaces/openshift-ingress-canary/pods/ingress-canary-dmmld/ingress-canary-dmmld.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875395629Z namespaces/openshift-ingress-canary/pods/ingress-canary-dmmld/serve-healthcheck-canary/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87540517Z namespaces/openshift-ingress-canary/pods/ingress-canary-dmmld/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875410481Z namespaces/openshift-ingress-canary/pods/ingress-canary-dmmld/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875456473Z namespaces/openshift-ingress-canary/pods/ingress-canary-dmmld/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87555999Z namespaces/openshift-ingress-canary/pods/ingress-canary-dmmld/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875638925Z namespaces/openshift-ingress-canary/pods/ingress-canary-dmmld/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875844247Z namespaces/openshift-ingress-canary/pods/ingress-canary-mw7nr/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87589115Z namespaces/openshift-ingress-canary/pods/ingress-canary-mw7nr/ingress-canary-mw7nr.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875972545Z namespaces/openshift-ingress-canary/pods/ingress-canary-mw7nr/serve-healthcheck-canary/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875980106Z namespaces/openshift-ingress-canary/pods/ingress-canary-mw7nr/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.875988526Z namespaces/openshift-ingress-canary/pods/ingress-canary-mw7nr/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87604126Z namespaces/openshift-ingress-canary/pods/ingress-canary-mw7nr/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876175308Z namespaces/openshift-ingress-canary/pods/ingress-canary-mw7nr/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876253143Z namespaces/openshift-ingress-canary/pods/ingress-canary-mw7nr/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876289355Z namespaces/openshift-ingress-canary/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876330218Z namespaces/openshift-ingress-canary/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876393441Z namespaces/openshift-ingress-canary/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876426073Z namespaces/openshift-ingress-canary/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876541911Z namespaces/openshift-ingress-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876595034Z namespaces/openshift-ingress-operator/openshift-ingress-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876669859Z namespaces/openshift-ingress-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87669964Z namespaces/openshift-ingress-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876777955Z namespaces/openshift-ingress-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876821118Z namespaces/openshift-ingress-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876906713Z namespaces/openshift-ingress-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.876984088Z namespaces/openshift-ingress-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.877070053Z namespaces/openshift-ingress-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.877136277Z namespaces/openshift-ingress-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.877186811Z namespaces/openshift-ingress-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.877264975Z namespaces/openshift-ingress-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.877303688Z namespaces/openshift-ingress-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.877390343Z namespaces/openshift-ingress-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.877427675Z namespaces/openshift-ingress-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.877471798Z namespaces/openshift-ingress-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.877557083Z namespaces/openshift-ingress-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.877632548Z namespaces/openshift-ingress-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87767199Z namespaces/openshift-ingress-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87847013Z namespaces/openshift-ingress-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.878556425Z namespaces/openshift-ingress-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.878646971Z namespaces/openshift-ingress-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.878731846Z namespaces/openshift-ingress-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.878851774Z namespaces/openshift-ingress-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.878984552Z namespaces/openshift-ingress-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879205005Z namespaces/openshift-ingress-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.87928008Z namespaces/openshift-ingress-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879326653Z namespaces/openshift-ingress-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879391817Z namespaces/openshift-ingress-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879424589Z namespaces/openshift-ingress-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879490143Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879503034Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/dnsrecords/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879546577Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/dnsrecords/default-wildcard.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879622561Z namespaces/openshift-ingress-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879722807Z namespaces/openshift-ingress-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879806962Z namespaces/openshift-ingress-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879897918Z namespaces/openshift-ingress-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.879954782Z namespaces/openshift-ingress-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880003915Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880056148Z namespaces/openshift-ingress-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880118172Z namespaces/openshift-ingress-operator/monitoring.coreos.com/prometheusrules/ingress-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880258501Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880292372Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors/ingress-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880368607Z namespaces/openshift-ingress-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880395919Z namespaces/openshift-ingress-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880471203Z namespaces/openshift-ingress-operator/operator.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880478784Z namespaces/openshift-ingress-operator/operator.openshift.io/ingresscontrollers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880512606Z namespaces/openshift-ingress-operator/operator.openshift.io/ingresscontrollers/default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880593591Z namespaces/openshift-ingress-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880616092Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880671986Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880818675Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.880992736Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.881144445Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.881370479Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.881566191Z namespaces/openshift-ingress-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.881608134Z namespaces/openshift-ingress-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.881669758Z namespaces/openshift-ingress-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.88170968Z namespaces/openshift-ingress-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.881810086Z namespaces/openshift-ingress/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.881843888Z namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882050161Z namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882092254Z namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882184189Z namespaces/openshift-ingress/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882224492Z namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882327698Z namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882584484Z namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882777466Z namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882821759Z namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882865432Z namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882931776Z namespaces/openshift-ingress/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.882972268Z namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883050773Z namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883111997Z namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883181141Z namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883290328Z namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883348692Z namespaces/openshift-ingress/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883356562Z namespaces/openshift-ingress/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883411265Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-ca-openshift-gateway.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883495781Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-deployment-openshift-gateway.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883578386Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-status-leader-openshift-gateway.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883683922Z namespaces/openshift-ingress/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.883720924Z namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.884150321Z namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.884277849Z namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.884487042Z namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.884565557Z namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.884827043Z namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.884955091Z namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885181245Z namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885274801Z namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885320643Z namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885414719Z namespaces/openshift-ingress/gateway.networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.88542561Z namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885458732Z namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/openshift-ai-inference.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885526546Z namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885547508Z namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885626233Z namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885652964Z namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885738579Z namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885831175Z namespaces/openshift-ingress/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885888219Z namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.88591864Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.885988825Z namespaces/openshift-ingress/monitoring.coreos.com/podmonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.886028077Z namespaces/openshift-ingress/monitoring.coreos.com/podmonitors/openshift-ai-inference-metrics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.886103792Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.88623323Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors/router-default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.886282573Z namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.886336186Z namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.886356567Z namespaces/openshift-ingress/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.88639424Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.886436863Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.886591372Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.886758923Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.886899021Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.887106774Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.887346699Z namespaces/openshift-ingress/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.88736203Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.887396922Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/istiod-openshift-gateway-94bb8fbfd-wnhkc.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.887511899Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.88752044Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.88752683Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.887568073Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.900385135Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.900454379Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-wnhkc/discovery/discovery/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.900500222Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.900560886Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/openshift-ai-inference-openshift-default-866675cbb8-qnxtj.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.900926099Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.90094244Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.90095014Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.900966181Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.901940811Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-866675cbb8-qnxtj/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.901997655Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902038068Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router-default-7454758546-dtbkw.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902195267Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902213548Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902220809Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.90224072Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902357937Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902426692Z namespaces/openshift-ingress/pods/router-default-7454758546-dtbkw/router/router/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902478185Z namespaces/openshift-ingress/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902516017Z namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902575441Z namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902612333Z namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902713289Z namespaces/openshift-insights/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902750181Z namespaces/openshift-insights/openshift-insights.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902833007Z namespaces/openshift-insights/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902870599Z namespaces/openshift-insights/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902948344Z namespaces/openshift-insights/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.902985906Z namespaces/openshift-insights/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903109174Z namespaces/openshift-insights/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903314316Z namespaces/openshift-insights/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903422163Z namespaces/openshift-insights/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903461156Z namespaces/openshift-insights/apps/deployments/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903505908Z namespaces/openshift-insights/apps/deployments/insights-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903578973Z namespaces/openshift-insights/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903627416Z namespaces/openshift-insights/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903683009Z namespaces/openshift-insights/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903722432Z namespaces/openshift-insights/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903807537Z namespaces/openshift-insights/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.90386305Z namespaces/openshift-insights/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903920994Z namespaces/openshift-insights/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.903990348Z namespaces/openshift-insights/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.904065953Z namespaces/openshift-insights/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.904137017Z namespaces/openshift-insights/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.905035573Z namespaces/openshift-insights/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.9051464Z namespaces/openshift-insights/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.905435568Z namespaces/openshift-insights/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.905573066Z namespaces/openshift-insights/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.905896296Z namespaces/openshift-insights/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.905974491Z namespaces/openshift-insights/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906145182Z namespaces/openshift-insights/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906238567Z namespaces/openshift-insights/core/configmaps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.90628419Z namespaces/openshift-insights/core/configmaps/service-ca-bundle.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906356965Z namespaces/openshift-insights/core/serviceaccounts/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906401757Z namespaces/openshift-insights/core/serviceaccounts/gather.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906501153Z namespaces/openshift-insights/core/serviceaccounts/operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906553457Z namespaces/openshift-insights/core/services/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906615351Z namespaces/openshift-insights/core/services/metrics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906663243Z namespaces/openshift-insights/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906730658Z namespaces/openshift-insights/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.9067721Z namespaces/openshift-insights/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906831194Z namespaces/openshift-insights/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906894278Z namespaces/openshift-insights/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.906938991Z namespaces/openshift-insights/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907029786Z namespaces/openshift-insights/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907110361Z namespaces/openshift-insights/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907191696Z namespaces/openshift-insights/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.90724357Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907349276Z namespaces/openshift-insights/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907370097Z namespaces/openshift-insights/monitoring.coreos.com/prometheusrules/insights-prometheus-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907449732Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907538338Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors/insights-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.90758018Z namespaces/openshift-insights/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907643944Z namespaces/openshift-insights/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907662245Z namespaces/openshift-insights/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907682857Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907753501Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.907955314Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.908194528Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.908338797Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.90854545Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.908785955Z namespaces/openshift-insights/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.908798056Z namespaces/openshift-insights/pods/insights-operator-68d9dd94cf-psmpz/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.908822387Z namespaces/openshift-insights/pods/insights-operator-68d9dd94cf-psmpz/insights-operator-68d9dd94cf-psmpz.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.908941164Z namespaces/openshift-insights/pods/insights-operator-68d9dd94cf-psmpz/insights-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.908953775Z namespaces/openshift-insights/pods/insights-operator-68d9dd94cf-psmpz/insights-operator/insights-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.908969576Z namespaces/openshift-insights/pods/insights-operator-68d9dd94cf-psmpz/insights-operator/insights-operator/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.908989507Z namespaces/openshift-insights/pods/insights-operator-68d9dd94cf-psmpz/insights-operator/insights-operator/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.909412344Z namespaces/openshift-insights/pods/insights-operator-68d9dd94cf-psmpz/insights-operator/insights-operator/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.909690631Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.909720593Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/insights-runtime-extractor-fsbjf.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.909822479Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.909829979Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/exporter/exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.9098341Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/exporter/exporter/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.909858251Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/exporter/exporter/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.909968988Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910037842Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/exporter/exporter/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910078965Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/extractor/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910086035Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/extractor/extractor/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910089886Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/extractor/extractor/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910112827Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/extractor/extractor/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910271787Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.91032182Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/extractor/extractor/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910358612Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910381044Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910385584Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910442487Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910523572Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910592937Z namespaces/openshift-insights/pods/insights-runtime-extractor-fsbjf/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910666401Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910676992Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/insights-runtime-extractor-ftsqx.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910767067Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910775288Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/exporter/exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910782358Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/exporter/exporter/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910839652Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/exporter/exporter/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910926047Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.910997342Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/exporter/exporter/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911034494Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/extractor/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911044654Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/extractor/extractor/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911048665Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/extractor/extractor/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911097468Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/extractor/extractor/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911224506Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.91129214Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/extractor/extractor/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911328342Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911516714Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911525384Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911552136Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911654492Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911733467Z namespaces/openshift-insights/pods/insights-runtime-extractor-ftsqx/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911760649Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911796431Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/insights-runtime-extractor-l9zbv.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911908818Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911922589Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/exporter/exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911929429Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/exporter/exporter/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.911959181Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/exporter/exporter/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912042966Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912140542Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/exporter/exporter/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912166734Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/extractor/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912180685Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/extractor/extractor/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912188105Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/extractor/extractor/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912231298Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/extractor/extractor/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912339765Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912392108Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/extractor/extractor/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912453502Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912462682Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912469493Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912473303Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912574979Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912642843Z namespaces/openshift-insights/pods/insights-runtime-extractor-l9zbv/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912676615Z namespaces/openshift-insights/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912705237Z namespaces/openshift-insights/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912765271Z namespaces/openshift-insights/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912812684Z namespaces/openshift-insights/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912873198Z namespaces/openshift-keda/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912884858Z namespaces/openshift-keda/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912890329Z namespaces/openshift-keda/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912897209Z namespaces/openshift-keda/coordination.k8s.io/leases/olm-operator.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.912997605Z namespaces/openshift-keda/coordination.k8s.io/leases/operator.keda.sh.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.913053779Z namespaces/openshift-keda/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.913063679Z namespaces/openshift-keda/monitoring.coreos.com/podmonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.913095781Z namespaces/openshift-keda/monitoring.coreos.com/podmonitors/keda-olm-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.913179777Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.913213539Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-admission-webhooks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.913305214Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-metrics-apiserver.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.913430482Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.913492146Z namespaces/openshift-keda/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.913499797Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.91355176Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.913696838Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.9138791Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/custom-metrics-autoscaler.v2.19.0-2.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914058231Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914288985Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914431844Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914601785Z namespaces/openshift-keda/operators.coreos.com/installplans/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914641337Z namespaces/openshift-keda/operators.coreos.com/installplans/install-bhc7s.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914777445Z namespaces/openshift-keda/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914790036Z namespaces/openshift-keda/operators.coreos.com/operatorconditions/custom-metrics-autoscaler.v2.19.0-2.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914840239Z namespaces/openshift-keda/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914882602Z namespaces/openshift-keda/operators.coreos.com/operatorgroups/openshift-keda.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914944746Z namespaces/openshift-keda/operators.coreos.com/subscriptions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.914975228Z namespaces/openshift-keda/operators.coreos.com/subscriptions/openshift-custom-metrics-autoscaler-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915084035Z namespaces/openshift-kube-apiserver-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915143958Z namespaces/openshift-kube-apiserver-operator/openshift-kube-apiserver-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915226273Z namespaces/openshift-kube-apiserver-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915278196Z namespaces/openshift-kube-apiserver-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915316309Z namespaces/openshift-kube-apiserver-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915358881Z namespaces/openshift-kube-apiserver-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915447327Z namespaces/openshift-kube-apiserver-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915536302Z namespaces/openshift-kube-apiserver-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915608407Z namespaces/openshift-kube-apiserver-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915675171Z namespaces/openshift-kube-apiserver-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915703113Z namespaces/openshift-kube-apiserver-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915773877Z namespaces/openshift-kube-apiserver-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915801179Z namespaces/openshift-kube-apiserver-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915900165Z namespaces/openshift-kube-apiserver-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915943568Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.915970589Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.916063395Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.916174862Z namespaces/openshift-kube-apiserver-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.916218554Z namespaces/openshift-kube-apiserver-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.916331712Z namespaces/openshift-kube-apiserver-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.916414997Z namespaces/openshift-kube-apiserver-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.916491422Z namespaces/openshift-kube-apiserver-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.916575187Z namespaces/openshift-kube-apiserver-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.916658652Z namespaces/openshift-kube-apiserver-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.916808681Z namespaces/openshift-kube-apiserver-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.916980052Z namespaces/openshift-kube-apiserver-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917021854Z namespaces/openshift-kube-apiserver-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917070117Z namespaces/openshift-kube-apiserver-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917137782Z namespaces/openshift-kube-apiserver-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917207306Z namespaces/openshift-kube-apiserver-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917219767Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.91727381Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917347074Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917428079Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917491573Z namespaces/openshift-kube-apiserver-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917533966Z namespaces/openshift-kube-apiserver-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917591659Z namespaces/openshift-kube-apiserver-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917649893Z namespaces/openshift-kube-apiserver-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917694996Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917711697Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917774851Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.917915699Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.918102171Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.918257041Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.918459113Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.918641735Z namespaces/openshift-kube-apiserver-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.918665886Z namespaces/openshift-kube-apiserver-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.918721729Z namespaces/openshift-kube-apiserver-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.918765512Z namespaces/openshift-kube-apiserver-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.91889642Z namespaces/openshift-kube-apiserver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.918939833Z namespaces/openshift-kube-apiserver/openshift-kube-apiserver.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919017888Z namespaces/openshift-kube-apiserver/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.91905854Z namespaces/openshift-kube-apiserver/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919141415Z namespaces/openshift-kube-apiserver/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919190838Z namespaces/openshift-kube-apiserver/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919276314Z namespaces/openshift-kube-apiserver/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919356099Z namespaces/openshift-kube-apiserver/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919439214Z namespaces/openshift-kube-apiserver/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919499657Z namespaces/openshift-kube-apiserver/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919531739Z namespaces/openshift-kube-apiserver/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919602574Z namespaces/openshift-kube-apiserver/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919637246Z namespaces/openshift-kube-apiserver/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919735532Z namespaces/openshift-kube-apiserver/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919767634Z namespaces/openshift-kube-apiserver/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919819897Z namespaces/openshift-kube-apiserver/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919902182Z namespaces/openshift-kube-apiserver/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.919969557Z namespaces/openshift-kube-apiserver/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920005709Z namespaces/openshift-kube-apiserver/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920118066Z namespaces/openshift-kube-apiserver/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920231043Z namespaces/openshift-kube-apiserver/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920307878Z namespaces/openshift-kube-apiserver/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920389492Z namespaces/openshift-kube-apiserver/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920472898Z namespaces/openshift-kube-apiserver/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920596475Z namespaces/openshift-kube-apiserver/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920748525Z namespaces/openshift-kube-apiserver/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920811029Z namespaces/openshift-kube-apiserver/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920857592Z namespaces/openshift-kube-apiserver/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920916275Z namespaces/openshift-kube-apiserver/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.920963998Z namespaces/openshift-kube-apiserver/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.921018682Z namespaces/openshift-kube-apiserver/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.921068694Z namespaces/openshift-kube-apiserver/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.921231445Z namespaces/openshift-kube-apiserver/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92132358Z namespaces/openshift-kube-apiserver/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.921396415Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.921423287Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.921502151Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.921534933Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/api-usage.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92163737Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/podsecurity.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.921758377Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92179803Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors/openshift-kube-apiserver.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.921863544Z namespaces/openshift-kube-apiserver/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.921901246Z namespaces/openshift-kube-apiserver/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92195726Z namespaces/openshift-kube-apiserver/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92196719Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.922004472Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.922175243Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.922368725Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.922491602Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.922705086Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.922883307Z namespaces/openshift-kube-apiserver/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.922913979Z namespaces/openshift-kube-apiserver/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.922970892Z namespaces/openshift-kube-apiserver/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923012875Z namespaces/openshift-kube-apiserver/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923079489Z namespaces/openshift-kube-controller-manager-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923117621Z namespaces/openshift-kube-controller-manager-operator/openshift-kube-controller-manager-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923211777Z namespaces/openshift-kube-controller-manager-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92325469Z namespaces/openshift-kube-controller-manager-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923310653Z namespaces/openshift-kube-controller-manager-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923340445Z namespaces/openshift-kube-controller-manager-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92342822Z namespaces/openshift-kube-controller-manager-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923507795Z namespaces/openshift-kube-controller-manager-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92358854Z namespaces/openshift-kube-controller-manager-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923635903Z namespaces/openshift-kube-controller-manager-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923664475Z namespaces/openshift-kube-controller-manager-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923724979Z namespaces/openshift-kube-controller-manager-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923754331Z namespaces/openshift-kube-controller-manager-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923845606Z namespaces/openshift-kube-controller-manager-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92389994Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.923949883Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.924051469Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.924119193Z namespaces/openshift-kube-controller-manager-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.924190247Z namespaces/openshift-kube-controller-manager-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.924298014Z namespaces/openshift-kube-controller-manager-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.924379739Z namespaces/openshift-kube-controller-manager-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.924461184Z namespaces/openshift-kube-controller-manager-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.924541199Z namespaces/openshift-kube-controller-manager-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.924631065Z namespaces/openshift-kube-controller-manager-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.924766283Z namespaces/openshift-kube-controller-manager-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.924975136Z namespaces/openshift-kube-controller-manager-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92504075Z namespaces/openshift-kube-controller-manager-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925083163Z namespaces/openshift-kube-controller-manager-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925139006Z namespaces/openshift-kube-controller-manager-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92520013Z namespaces/openshift-kube-controller-manager-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925254744Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925299106Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925382281Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925466357Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925514729Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925556582Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925613816Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925642088Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors/kube-controller-manager-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925709142Z namespaces/openshift-kube-controller-manager-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925737253Z namespaces/openshift-kube-controller-manager-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925818508Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.925834969Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92584786Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92599958Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.926206252Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.926346491Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.926532483Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.926717434Z namespaces/openshift-kube-controller-manager-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.926755716Z namespaces/openshift-kube-controller-manager-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92681234Z namespaces/openshift-kube-controller-manager-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.926848752Z namespaces/openshift-kube-controller-manager-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.926922137Z namespaces/openshift-kube-controller-manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92697517Z namespaces/openshift-kube-controller-manager/openshift-kube-controller-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927024883Z namespaces/openshift-kube-controller-manager/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927058385Z namespaces/openshift-kube-controller-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92713988Z namespaces/openshift-kube-controller-manager/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927177552Z namespaces/openshift-kube-controller-manager/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927287019Z namespaces/openshift-kube-controller-manager/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927354703Z namespaces/openshift-kube-controller-manager/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927431778Z namespaces/openshift-kube-controller-manager/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927478231Z namespaces/openshift-kube-controller-manager/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927519073Z namespaces/openshift-kube-controller-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927577977Z namespaces/openshift-kube-controller-manager/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927594798Z namespaces/openshift-kube-controller-manager/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927702055Z namespaces/openshift-kube-controller-manager/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927751608Z namespaces/openshift-kube-controller-manager/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927770959Z namespaces/openshift-kube-controller-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927874335Z namespaces/openshift-kube-controller-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927912198Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927921618Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.927956851Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/leases/cluster-policy-controller-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.928026335Z namespaces/openshift-kube-controller-manager/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.928062727Z namespaces/openshift-kube-controller-manager/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.928204786Z namespaces/openshift-kube-controller-manager/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.928285811Z namespaces/openshift-kube-controller-manager/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.928570078Z namespaces/openshift-kube-controller-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.928643983Z namespaces/openshift-kube-controller-manager/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.928724788Z namespaces/openshift-kube-controller-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.928857576Z namespaces/openshift-kube-controller-manager/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929022877Z namespaces/openshift-kube-controller-manager/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929068519Z namespaces/openshift-kube-controller-manager/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929112122Z namespaces/openshift-kube-controller-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929299284Z namespaces/openshift-kube-controller-manager/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929336286Z namespaces/openshift-kube-controller-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92940069Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929459433Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929534188Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929628574Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929661466Z namespaces/openshift-kube-controller-manager/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929709849Z namespaces/openshift-kube-controller-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929767803Z namespaces/openshift-kube-controller-manager/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929788744Z namespaces/openshift-kube-controller-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929852068Z namespaces/openshift-kube-controller-manager/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.929863569Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.92988009Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.930032819Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.930235321Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93037274Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.930577203Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.930765484Z namespaces/openshift-kube-controller-manager/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.930786276Z namespaces/openshift-kube-controller-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93085796Z namespaces/openshift-kube-controller-manager/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.930887002Z namespaces/openshift-kube-controller-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.930953256Z namespaces/openshift-kube-scheduler-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.930998709Z namespaces/openshift-kube-scheduler-operator/openshift-kube-scheduler-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931061803Z namespaces/openshift-kube-scheduler-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931090345Z namespaces/openshift-kube-scheduler-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93117718Z namespaces/openshift-kube-scheduler-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931211182Z namespaces/openshift-kube-scheduler-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931302567Z namespaces/openshift-kube-scheduler-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931386123Z namespaces/openshift-kube-scheduler-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931459187Z namespaces/openshift-kube-scheduler-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93150656Z namespaces/openshift-kube-scheduler-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931545853Z namespaces/openshift-kube-scheduler-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931592545Z namespaces/openshift-kube-scheduler-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931637878Z namespaces/openshift-kube-scheduler-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931927796Z namespaces/openshift-kube-scheduler-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.931973579Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.932005711Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.932092967Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.932176022Z namespaces/openshift-kube-scheduler-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.932231725Z namespaces/openshift-kube-scheduler-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.932341562Z namespaces/openshift-kube-scheduler-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.932421537Z namespaces/openshift-kube-scheduler-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.932506662Z namespaces/openshift-kube-scheduler-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.932587197Z namespaces/openshift-kube-scheduler-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.932666642Z namespaces/openshift-kube-scheduler-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.932788179Z namespaces/openshift-kube-scheduler-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93295838Z namespaces/openshift-kube-scheduler-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933008613Z namespaces/openshift-kube-scheduler-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933037815Z namespaces/openshift-kube-scheduler-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933098749Z namespaces/openshift-kube-scheduler-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933166983Z namespaces/openshift-kube-scheduler-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933220136Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933267619Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933358935Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933437179Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933484713Z namespaces/openshift-kube-scheduler-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933562337Z namespaces/openshift-kube-scheduler-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933582829Z namespaces/openshift-kube-scheduler-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933645243Z namespaces/openshift-kube-scheduler-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933688475Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933699706Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933753389Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.933888138Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.934074839Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.934298263Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.934498045Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.934692677Z namespaces/openshift-kube-scheduler-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93473195Z namespaces/openshift-kube-scheduler-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.934786743Z namespaces/openshift-kube-scheduler-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.934819525Z namespaces/openshift-kube-scheduler-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.934919211Z namespaces/openshift-kube-scheduler/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.934936962Z namespaces/openshift-kube-scheduler/openshift-kube-scheduler.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935021368Z namespaces/openshift-kube-scheduler/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935035729Z namespaces/openshift-kube-scheduler/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935096252Z namespaces/openshift-kube-scheduler/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935149776Z namespaces/openshift-kube-scheduler/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935247222Z namespaces/openshift-kube-scheduler/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935324476Z namespaces/openshift-kube-scheduler/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935404521Z namespaces/openshift-kube-scheduler/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935457484Z namespaces/openshift-kube-scheduler/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935486536Z namespaces/openshift-kube-scheduler/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93554887Z namespaces/openshift-kube-scheduler/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935584612Z namespaces/openshift-kube-scheduler/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935678438Z namespaces/openshift-kube-scheduler/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93569787Z namespaces/openshift-kube-scheduler/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935760873Z namespaces/openshift-kube-scheduler/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935843348Z namespaces/openshift-kube-scheduler/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935885461Z namespaces/openshift-kube-scheduler/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.935923973Z namespaces/openshift-kube-scheduler/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93603351Z namespaces/openshift-kube-scheduler/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936111945Z namespaces/openshift-kube-scheduler/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936229912Z namespaces/openshift-kube-scheduler/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936317638Z namespaces/openshift-kube-scheduler/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936410523Z namespaces/openshift-kube-scheduler/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936562463Z namespaces/openshift-kube-scheduler/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936730463Z namespaces/openshift-kube-scheduler/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936779916Z namespaces/openshift-kube-scheduler/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936809738Z namespaces/openshift-kube-scheduler/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936869212Z namespaces/openshift-kube-scheduler/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936902934Z namespaces/openshift-kube-scheduler/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.936964458Z namespaces/openshift-kube-scheduler/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937013651Z namespaces/openshift-kube-scheduler/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937090676Z namespaces/openshift-kube-scheduler/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937190062Z namespaces/openshift-kube-scheduler/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937228984Z namespaces/openshift-kube-scheduler/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937276507Z namespaces/openshift-kube-scheduler/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937309949Z namespaces/openshift-kube-scheduler/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937362032Z namespaces/openshift-kube-scheduler/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937407845Z namespaces/openshift-kube-scheduler/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937414805Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937460799Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937610178Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937797749Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.937933578Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938142431Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938351183Z namespaces/openshift-kube-scheduler/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938390516Z namespaces/openshift-kube-scheduler/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938441429Z namespaces/openshift-kube-scheduler/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938472511Z namespaces/openshift-kube-scheduler/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938546186Z namespaces/openshift-kube-storage-version-migrator-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938592998Z namespaces/openshift-kube-storage-version-migrator-operator/openshift-kube-storage-version-migrator-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938660613Z namespaces/openshift-kube-storage-version-migrator-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938675974Z namespaces/openshift-kube-storage-version-migrator-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938743508Z namespaces/openshift-kube-storage-version-migrator-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938791541Z namespaces/openshift-kube-storage-version-migrator-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938875726Z namespaces/openshift-kube-storage-version-migrator-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.938996903Z namespaces/openshift-kube-storage-version-migrator-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93910078Z namespaces/openshift-kube-storage-version-migrator-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939171764Z namespaces/openshift-kube-storage-version-migrator-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939208607Z namespaces/openshift-kube-storage-version-migrator-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93926713Z namespaces/openshift-kube-storage-version-migrator-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939294572Z namespaces/openshift-kube-storage-version-migrator-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939389238Z namespaces/openshift-kube-storage-version-migrator-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.93942863Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939480313Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939561568Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939615622Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939631023Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939643054Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/leases/openshift-kube-storage-version-migrator-operator-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939718048Z namespaces/openshift-kube-storage-version-migrator-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939769661Z namespaces/openshift-kube-storage-version-migrator-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939869857Z namespaces/openshift-kube-storage-version-migrator-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.939954853Z namespaces/openshift-kube-storage-version-migrator-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.940167686Z namespaces/openshift-kube-storage-version-migrator-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.940267722Z namespaces/openshift-kube-storage-version-migrator-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.940426322Z namespaces/openshift-kube-storage-version-migrator-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.94055696Z namespaces/openshift-kube-storage-version-migrator-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.940736041Z namespaces/openshift-kube-storage-version-migrator-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.940796835Z namespaces/openshift-kube-storage-version-migrator-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.940839457Z namespaces/openshift-kube-storage-version-migrator-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.940895161Z namespaces/openshift-kube-storage-version-migrator-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.940927203Z namespaces/openshift-kube-storage-version-migrator-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.940987597Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941017368Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941113524Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941215421Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941243382Z namespaces/openshift-kube-storage-version-migrator-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941304116Z namespaces/openshift-kube-storage-version-migrator-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.94136208Z namespaces/openshift-kube-storage-version-migrator-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941381281Z namespaces/openshift-kube-storage-version-migrator-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941449995Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941461886Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941492738Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941646997Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941826049Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.941966187Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942236144Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942430076Z namespaces/openshift-kube-storage-version-migrator-operator/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942438276Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-799cb7f958-lk6hb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942482359Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-799cb7f958-lk6hb/kube-storage-version-migrator-operator-799cb7f958-lk6hb.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942604787Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-799cb7f958-lk6hb/kube-storage-version-migrator-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942613167Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-799cb7f958-lk6hb/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942622398Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-799cb7f958-lk6hb/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942673991Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-799cb7f958-lk6hb/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942792328Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-799cb7f958-lk6hb/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942914066Z namespaces/openshift-kube-storage-version-migrator-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.942939007Z namespaces/openshift-kube-storage-version-migrator-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.94299035Z namespaces/openshift-kube-storage-version-migrator-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943038853Z namespaces/openshift-kube-storage-version-migrator-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943092877Z namespaces/openshift-kube-storage-version-migrator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.94313919Z namespaces/openshift-kube-storage-version-migrator/openshift-kube-storage-version-migrator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943218544Z namespaces/openshift-kube-storage-version-migrator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943254687Z namespaces/openshift-kube-storage-version-migrator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.9433129Z namespaces/openshift-kube-storage-version-migrator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943355363Z namespaces/openshift-kube-storage-version-migrator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943439898Z namespaces/openshift-kube-storage-version-migrator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943577297Z namespaces/openshift-kube-storage-version-migrator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943678373Z namespaces/openshift-kube-storage-version-migrator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943725516Z namespaces/openshift-kube-storage-version-migrator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943768959Z namespaces/openshift-kube-storage-version-migrator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943822762Z namespaces/openshift-kube-storage-version-migrator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943864655Z namespaces/openshift-kube-storage-version-migrator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943942329Z namespaces/openshift-kube-storage-version-migrator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.943997533Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.944042106Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.944149972Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.944210446Z namespaces/openshift-kube-storage-version-migrator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.944257579Z namespaces/openshift-kube-storage-version-migrator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.944365106Z namespaces/openshift-kube-storage-version-migrator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.94444468Z namespaces/openshift-kube-storage-version-migrator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.944563388Z namespaces/openshift-kube-storage-version-migrator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.944644613Z namespaces/openshift-kube-storage-version-migrator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.94475845Z namespaces/openshift-kube-storage-version-migrator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.944886228Z namespaces/openshift-kube-storage-version-migrator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945048548Z namespaces/openshift-kube-storage-version-migrator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945100881Z namespaces/openshift-kube-storage-version-migrator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945166535Z namespaces/openshift-kube-storage-version-migrator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945218588Z namespaces/openshift-kube-storage-version-migrator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945263571Z namespaces/openshift-kube-storage-version-migrator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945312334Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945357617Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945446742Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945526827Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.94557435Z namespaces/openshift-kube-storage-version-migrator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945620673Z namespaces/openshift-kube-storage-version-migrator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945668266Z namespaces/openshift-kube-storage-version-migrator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945696728Z namespaces/openshift-kube-storage-version-migrator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945765202Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945776113Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945829926Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.945987146Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.946176958Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.946321147Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.94653371Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.946711111Z namespaces/openshift-kube-storage-version-migrator/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.946724081Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.946734302Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/migrator-7f59b54f6c-5fthk.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.946829528Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/graceful-termination/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.946840428Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/graceful-termination/graceful-termination/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.946846989Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/graceful-termination/graceful-termination/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.946889462Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/graceful-termination/graceful-termination/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.946984577Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/graceful-termination/graceful-termination/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947054022Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/graceful-termination/graceful-termination/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947091314Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/migrator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947100645Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/migrator/migrator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947106985Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/migrator/migrator/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947143747Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/migrator/migrator/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.94735216Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/migrator/migrator/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947422955Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7f59b54f6c-5fthk/migrator/migrator/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947441776Z namespaces/openshift-kube-storage-version-migrator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947490679Z namespaces/openshift-kube-storage-version-migrator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947544232Z namespaces/openshift-kube-storage-version-migrator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947575214Z namespaces/openshift-kube-storage-version-migrator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947644248Z namespaces/openshift-lws-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947684031Z namespaces/openshift-lws-operator/openshift-lws-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947733384Z namespaces/openshift-lws-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947789407Z namespaces/openshift-lws-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947859262Z namespaces/openshift-lws-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947896934Z namespaces/openshift-lws-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.947960988Z namespaces/openshift-lws-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948120688Z namespaces/openshift-lws-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948396445Z namespaces/openshift-lws-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948462209Z namespaces/openshift-lws-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948530563Z namespaces/openshift-lws-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948571326Z namespaces/openshift-lws-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948591517Z namespaces/openshift-lws-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948696813Z namespaces/openshift-lws-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948711924Z namespaces/openshift-lws-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948773418Z namespaces/openshift-lws-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948870784Z namespaces/openshift-lws-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948888725Z namespaces/openshift-lws-operator/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948894436Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.948945759Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/b8b2488c.x-k8s.io.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.949035354Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/openshift-lws-operator-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.949096468Z namespaces/openshift-lws-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.949151271Z namespaces/openshift-lws-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.94928626Z namespaces/openshift-lws-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.949385286Z namespaces/openshift-lws-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.949686395Z namespaces/openshift-lws-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.949764449Z namespaces/openshift-lws-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.949943291Z namespaces/openshift-lws-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950073819Z namespaces/openshift-lws-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950351836Z namespaces/openshift-lws-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950398599Z namespaces/openshift-lws-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950457902Z namespaces/openshift-lws-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950521966Z namespaces/openshift-lws-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950541908Z namespaces/openshift-lws-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950599751Z namespaces/openshift-lws-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950648454Z namespaces/openshift-lws-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950729289Z namespaces/openshift-lws-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950820825Z namespaces/openshift-lws-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950862237Z namespaces/openshift-lws-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95090664Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.950978855Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.951017927Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors/lws-controller-manager-metrics-monitor.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.9510664Z namespaces/openshift-lws-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.951096022Z namespaces/openshift-lws-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.951177047Z namespaces/openshift-lws-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.951191498Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95122885Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.951380619Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95155729Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.951690288Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/leader-worker-set.v1.0.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.951934924Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952143777Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95235622Z namespaces/openshift-lws-operator/operators.coreos.com/installplans/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952404573Z namespaces/openshift-lws-operator/operators.coreos.com/installplans/install-892k5.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952488428Z namespaces/openshift-lws-operator/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95253118Z namespaces/openshift-lws-operator/operators.coreos.com/operatorconditions/leader-worker-set.v1.0.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952582704Z namespaces/openshift-lws-operator/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952626486Z namespaces/openshift-lws-operator/operators.coreos.com/operatorgroups/leader-worker-set.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95268453Z namespaces/openshift-lws-operator/operators.coreos.com/subscriptions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952726063Z namespaces/openshift-lws-operator/operators.coreos.com/subscriptions/leader-worker-set.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952800077Z namespaces/openshift-lws-operator/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952808078Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-ln2vp/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95285166Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-ln2vp/lws-controller-manager-5c87b7c6dd-ln2vp.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952934296Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-ln2vp/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952941496Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-ln2vp/manager/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952945716Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-ln2vp/manager/manager/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.952978708Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-ln2vp/manager/manager/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953156069Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-ln2vp/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953248635Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-ln2vp/manager/manager/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953306159Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-v85f7/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953367012Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-v85f7/lws-controller-manager-5c87b7c6dd-v85f7.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953426006Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-v85f7/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953434567Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-v85f7/manager/manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953447157Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-v85f7/manager/manager/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953502591Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-v85f7/manager/manager/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953720774Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-v85f7/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953797159Z namespaces/openshift-lws-operator/pods/lws-controller-manager-5c87b7c6dd-v85f7/manager/manager/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953851852Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-2hxnc/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953892695Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-2hxnc/openshift-lws-operator-fd8ccff4c-2hxnc.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953986631Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-2hxnc/openshift-lws-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.953995851Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-2hxnc/openshift-lws-operator/openshift-lws-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954022623Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-2hxnc/openshift-lws-operator/openshift-lws-operator/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954088207Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-2hxnc/openshift-lws-operator/openshift-lws-operator/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954275079Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-2hxnc/openshift-lws-operator/openshift-lws-operator/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954345763Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-2hxnc/openshift-lws-operator/openshift-lws-operator/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954381075Z namespaces/openshift-lws-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954428828Z namespaces/openshift-lws-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954482521Z namespaces/openshift-lws-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954522924Z namespaces/openshift-lws-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954571387Z namespaces/openshift-machine-api/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954578067Z namespaces/openshift-machine-api/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954582267Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954640271Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95477978Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.954965931Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.955101959Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.955343585Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.955530466Z namespaces/openshift-machine-config-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.955539057Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.955543277Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.955574019Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/machine-config-controller.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.955674995Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/machine-config-daemon.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95575872Z namespaces/openshift-machine-config-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.955768111Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.955830985Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.955994975Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.956222239Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.956355057Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.956552319Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.956750032Z namespaces/openshift-marketplace/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.956756882Z namespaces/openshift-marketplace/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.956760722Z namespaces/openshift-marketplace/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.956810875Z namespaces/openshift-marketplace/monitoring.coreos.com/prometheusrules/marketplace-alert-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95688521Z namespaces/openshift-marketplace/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.956897601Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.956926202Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/certified-operators.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.957030679Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/community-operators.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.957116924Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/redhat-marketplace.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.957230501Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/redhat-operators.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.957296075Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95736687Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.957525689Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.957703511Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.957836979Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.958042332Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.95834276Z namespaces/openshift-monitoring/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.958386053Z namespaces/openshift-monitoring/openshift-monitoring.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.958480628Z namespaces/openshift-monitoring/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.9584964Z namespaces/openshift-monitoring/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.958578114Z namespaces/openshift-monitoring/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.958623997Z namespaces/openshift-monitoring/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.958763636Z namespaces/openshift-monitoring/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.959263047Z namespaces/openshift-monitoring/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.959612638Z namespaces/openshift-monitoring/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.959849123Z namespaces/openshift-monitoring/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.959887246Z namespaces/openshift-monitoring/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.959935118Z namespaces/openshift-monitoring/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.959985872Z namespaces/openshift-monitoring/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.960071227Z namespaces/openshift-monitoring/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.960181804Z namespaces/openshift-monitoring/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.960217976Z namespaces/openshift-monitoring/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.960316712Z namespaces/openshift-monitoring/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.960360935Z namespaces/openshift-monitoring/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.960402858Z namespaces/openshift-monitoring/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.963564003Z namespaces/openshift-monitoring/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.963705152Z namespaces/openshift-monitoring/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.964766697Z namespaces/openshift-monitoring/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.964846402Z namespaces/openshift-monitoring/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.965753868Z namespaces/openshift-monitoring/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.965872626Z namespaces/openshift-monitoring/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.966538997Z namespaces/openshift-monitoring/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.966704337Z namespaces/openshift-monitoring/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.966737269Z namespaces/openshift-monitoring/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.966887708Z namespaces/openshift-monitoring/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.966929931Z namespaces/openshift-monitoring/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.966979014Z namespaces/openshift-monitoring/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967021707Z namespaces/openshift-monitoring/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967105582Z namespaces/openshift-monitoring/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967208118Z namespaces/openshift-monitoring/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967261071Z namespaces/openshift-monitoring/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967294624Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967425522Z namespaces/openshift-monitoring/monitoring.coreos.com/alertmanagers/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967466204Z namespaces/openshift-monitoring/monitoring.coreos.com/alertmanagers/main.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967548449Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheuses/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967594672Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheuses/k8s.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967693938Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967744241Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/alertmanager-main-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.967849028Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/cluster-monitoring-operator-prometheus-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.968017378Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/kube-state-metrics-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.968109284Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/kubernetes-monitoring-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.968397412Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/node-exporter-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.968535721Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-k8s-prometheus-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.968669699Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-k8s-thanos-sidecar-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.968762455Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-operator-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.968868641Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/telemetry.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.968998749Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/thanos-querier.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.969065863Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.969114066Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/alertmanager-main.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.969234554Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/cluster-monitoring-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.969316319Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/kube-state-metrics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.969396984Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/kubelet.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.969507401Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/metrics-server.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.969586205Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/node-exporter.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.969735375Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/openshift-state-metrics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.969832571Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/prometheus-k8s.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.969932537Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/prometheus-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970010282Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/telemeter-client.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970102087Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/thanos-querier.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970230055Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/thanos-sidecar.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970295879Z namespaces/openshift-monitoring/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970334132Z namespaces/openshift-monitoring/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970384375Z namespaces/openshift-monitoring/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970392355Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970429698Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970640531Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970848593Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.970991132Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.971228207Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97143735Z namespaces/openshift-monitoring/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.971479303Z namespaces/openshift-monitoring/operators.coreos.com/operatorgroups/openshift-cluster-monitoring.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.971562988Z namespaces/openshift-monitoring/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.971571018Z namespaces/openshift-monitoring/pods/alertmanager-main-0/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.971634822Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager-main-0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.971801492Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.971808883Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.971812913Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.971851726Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.971951502Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972022136Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972052388Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972059939Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972069119Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972119512Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97224169Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972306274Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972350606Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972360027Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972374868Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972397769Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972482925Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97256488Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972582981Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972594092Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972598052Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972643475Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.9727355Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972804905Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972849507Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972856758Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972860888Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972905071Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.972999517Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973069511Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973110193Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973118084Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973135725Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973180018Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973274414Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973344958Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973386661Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973394191Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973400501Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973439364Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97353611Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973604324Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973664608Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-5f7f494dd4-gfs8k/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973711041Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-5f7f494dd4-gfs8k/cluster-monitoring-operator-5f7f494dd4-gfs8k.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973793866Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-5f7f494dd4-gfs8k/cluster-monitoring-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973802226Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-5f7f494dd4-gfs8k/cluster-monitoring-operator/cluster-monitoring-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973809117Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-5f7f494dd4-gfs8k/cluster-monitoring-operator/cluster-monitoring-operator/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.973850069Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-5f7f494dd4-gfs8k/cluster-monitoring-operator/cluster-monitoring-operator/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.974570314Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-5f7f494dd4-gfs8k/cluster-monitoring-operator/cluster-monitoring-operator/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.974639338Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-5f7f494dd4-gfs8k/cluster-monitoring-operator/cluster-monitoring-operator/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97467574Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.974720053Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-state-metrics-785778ff74-cs72g.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.974822589Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-main/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97482982Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-main/kube-rbac-proxy-main/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97483525Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.974871082Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.974962188Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975031902Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975068855Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-self/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975087246Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-self/kube-rbac-proxy-self/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975092956Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975114697Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975222064Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975291448Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97531495Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-state-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975324881Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-state-metrics/kube-state-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975328961Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-state-metrics/kube-state-metrics/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975394165Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-state-metrics/kube-state-metrics/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975587747Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-state-metrics/kube-state-metrics/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975659951Z namespaces/openshift-monitoring/pods/kube-state-metrics-785778ff74-cs72g/kube-state-metrics/kube-state-metrics/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975677992Z namespaces/openshift-monitoring/pods/metrics-server-6587bc5797-tkxtf/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975732246Z namespaces/openshift-monitoring/pods/metrics-server-6587bc5797-tkxtf/metrics-server-6587bc5797-tkxtf.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975832472Z namespaces/openshift-monitoring/pods/metrics-server-6587bc5797-tkxtf/metrics-server/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975842842Z namespaces/openshift-monitoring/pods/metrics-server-6587bc5797-tkxtf/metrics-server/metrics-server/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975847653Z namespaces/openshift-monitoring/pods/metrics-server-6587bc5797-tkxtf/metrics-server/metrics-server/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.975879075Z namespaces/openshift-monitoring/pods/metrics-server-6587bc5797-tkxtf/metrics-server/metrics-server/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97597403Z namespaces/openshift-monitoring/pods/metrics-server-6587bc5797-tkxtf/metrics-server/metrics-server/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976033864Z namespaces/openshift-monitoring/pods/metrics-server-6587bc5797-tkxtf/metrics-server/metrics-server/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976071116Z namespaces/openshift-monitoring/pods/monitoring-plugin-5f68df7f58-pwbl4/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976141091Z namespaces/openshift-monitoring/pods/monitoring-plugin-5f68df7f58-pwbl4/monitoring-plugin-5f68df7f58-pwbl4.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976225576Z namespaces/openshift-monitoring/pods/monitoring-plugin-5f68df7f58-pwbl4/monitoring-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976233666Z namespaces/openshift-monitoring/pods/monitoring-plugin-5f68df7f58-pwbl4/monitoring-plugin/monitoring-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976237577Z namespaces/openshift-monitoring/pods/monitoring-plugin-5f68df7f58-pwbl4/monitoring-plugin/monitoring-plugin/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97628846Z namespaces/openshift-monitoring/pods/monitoring-plugin-5f68df7f58-pwbl4/monitoring-plugin/monitoring-plugin/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976732477Z namespaces/openshift-monitoring/pods/monitoring-plugin-5f68df7f58-pwbl4/monitoring-plugin/monitoring-plugin/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976785551Z namespaces/openshift-monitoring/pods/monitoring-plugin-5f68df7f58-pwbl4/monitoring-plugin/monitoring-plugin/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976799302Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976809582Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/node-exporter-j6rrx.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976825193Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/init-textfile/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976840834Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/init-textfile/init-textfile/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976845864Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/init-textfile/init-textfile/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976858395Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.976866786Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977147063Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977215557Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977235259Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977241259Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977246649Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97726072Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977277201Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977405299Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/node-exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97741604Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/node-exporter/node-exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97742115Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/node-exporter/node-exporter/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977426591Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977627503Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977657345Z namespaces/openshift-monitoring/pods/node-exporter-j6rrx/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.977663065Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97774652Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/node-exporter-qnm77.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97807335Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/init-textfile/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978084851Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/init-textfile/init-textfile/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978090121Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/init-textfile/init-textfile/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978105592Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97823432Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978313985Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978359778Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978373729Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97838311Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97839038Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978499467Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978566871Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978611914Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/node-exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978622334Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/node-exporter/node-exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978627025Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/node-exporter/node-exporter/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978657136Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978768203Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978836508Z namespaces/openshift-monitoring/pods/node-exporter-qnm77/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978882421Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.978921443Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/node-exporter-s4xzg.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979021839Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/init-textfile/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97903062Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/init-textfile/init-textfile/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97903575Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/init-textfile/init-textfile/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979068002Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979169208Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979257974Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979299076Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979308807Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979313397Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97935321Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979442615Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.97951396Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979552672Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/node-exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979561723Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/node-exporter/node-exporter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979569243Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/node-exporter/node-exporter/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979609015Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979724433Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979790737Z namespaces/openshift-monitoring/pods/node-exporter-s4xzg/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979837769Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.979868271Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/openshift-state-metrics-84bd68b896-bf8kt.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98001079Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-main/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980021651Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-main/kube-rbac-proxy-main/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980026081Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980064303Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980196212Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980288427Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980322899Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-self/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98032931Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-self/kube-rbac-proxy-self/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98033361Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980376973Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980470669Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980543073Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980584826Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/openshift-state-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980593766Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/openshift-state-metrics/openshift-state-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980600047Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/openshift-state-metrics/openshift-state-metrics/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980643789Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/openshift-state-metrics/openshift-state-metrics/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980753536Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/openshift-state-metrics/openshift-state-metrics/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98082038Z namespaces/openshift-monitoring/pods/openshift-state-metrics-84bd68b896-bf8kt/openshift-state-metrics/openshift-state-metrics/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980869533Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.980932157Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus-k8s-0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981114328Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98114055Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981154391Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981207694Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981310971Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981381855Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981407877Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981412777Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981419767Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981478781Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981583268Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981644411Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981679994Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981688934Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981693034Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981726806Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981819052Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981887346Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981925319Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981937379Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98194382Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.981973622Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.982059197Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.982158103Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.982204246Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.982218907Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.982227397Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98226283Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.982361706Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98242797Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.982479083Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.982491674Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.982496334Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.982524526Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98307511Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983248071Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983284683Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983292303Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983301854Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983343236Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983455653Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983530718Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983574241Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983620783Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/prometheus-operator-64785b6675-pkvsz.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983715219Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98372504Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98372944Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983756802Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983868579Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983938153Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983981136Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/prometheus-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.983989976Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/prometheus-operator/prometheus-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984002307Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/prometheus-operator/prometheus-operator/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98404486Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/prometheus-operator/prometheus-operator/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984354749Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/prometheus-operator/prometheus-operator/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984425373Z namespaces/openshift-monitoring/pods/prometheus-operator-64785b6675-pkvsz/prometheus-operator/prometheus-operator/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984454685Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-77d98bc786-84sw4/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984504578Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-77d98bc786-84sw4/prometheus-operator-admission-webhook-77d98bc786-84sw4.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984631356Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-77d98bc786-84sw4/prometheus-operator-admission-webhook/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984641127Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-77d98bc786-84sw4/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984650767Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-77d98bc786-84sw4/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984680719Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-77d98bc786-84sw4/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984778175Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-77d98bc786-84sw4/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984842739Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-77d98bc786-84sw4/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984892612Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.984927324Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/telemeter-client-575fc68c48-9bwv8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985080474Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985090254Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985101785Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985156338Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985263575Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985328679Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985372552Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/reload/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985379872Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/reload/reload/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985387413Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/reload/reload/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985442666Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/reload/reload/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985538952Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/reload/reload/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985615167Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/reload/reload/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985659409Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/telemeter-client/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98567245Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/telemeter-client/telemeter-client/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985680041Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/telemeter-client/telemeter-client/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985700042Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/telemeter-client/telemeter-client/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985869553Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/telemeter-client/telemeter-client/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.985943777Z namespaces/openshift-monitoring/pods/telemeter-client-575fc68c48-9bwv8/telemeter-client/telemeter-client/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98598494Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986078606Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/thanos-querier-7f9d78dc5c-hhzqd.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986226605Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986240636Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986245246Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98630905Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986412486Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986482251Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986585737Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-rules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986596998Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-rules/kube-rbac-proxy-rules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986604188Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98664234Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986739366Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986806111Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986846833Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-web/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986858634Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986863094Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986889866Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.986981691Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987047255Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987089408Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987096378Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987100769Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987152502Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98728051Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987352554Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987398967Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/prom-label-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987408308Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/prom-label-proxy/prom-label-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987419918Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/prom-label-proxy/prom-label-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.98744708Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/prom-label-proxy/prom-label-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987539066Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/prom-label-proxy/prom-label-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.9876024Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/prom-label-proxy/prom-label-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987649703Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/thanos-query/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987661203Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/thanos-query/thanos-query/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987667084Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/thanos-query/thanos-query/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987691635Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/thanos-query/thanos-query/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987807942Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/thanos-query/thanos-query/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987882617Z namespaces/openshift-monitoring/pods/thanos-querier-7f9d78dc5c-hhzqd/thanos-query/thanos-query/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987906219Z namespaces/openshift-monitoring/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.987959312Z namespaces/openshift-monitoring/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.988024386Z namespaces/openshift-monitoring/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.988064558Z namespaces/openshift-monitoring/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.988554939Z namespaces/openshift-multus/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.988609462Z namespaces/openshift-multus/openshift-multus.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.988681927Z namespaces/openshift-multus/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.988698797Z namespaces/openshift-multus/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.988763702Z namespaces/openshift-multus/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.988802374Z namespaces/openshift-multus/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989039259Z namespaces/openshift-multus/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989112883Z namespaces/openshift-multus/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989241251Z namespaces/openshift-multus/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989283434Z namespaces/openshift-multus/apps/daemonsets/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989327416Z namespaces/openshift-multus/apps/daemonsets/multus-additional-cni-plugins.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989465805Z namespaces/openshift-multus/apps/daemonsets/multus.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989583642Z namespaces/openshift-multus/apps/daemonsets/network-metrics-daemon.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989657347Z namespaces/openshift-multus/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989737062Z namespaces/openshift-multus/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989794305Z namespaces/openshift-multus/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989821317Z namespaces/openshift-multus/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989950815Z namespaces/openshift-multus/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.989966326Z namespaces/openshift-multus/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.990058642Z namespaces/openshift-multus/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.990073093Z namespaces/openshift-multus/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.990264784Z namespaces/openshift-multus/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.990310647Z namespaces/openshift-multus/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.990457356Z namespaces/openshift-multus/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.990545302Z namespaces/openshift-multus/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.991010961Z namespaces/openshift-multus/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.991147349Z namespaces/openshift-multus/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.991791749Z namespaces/openshift-multus/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.991881184Z namespaces/openshift-multus/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992013062Z namespaces/openshift-multus/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992068706Z namespaces/openshift-multus/core/configmaps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99214215Z namespaces/openshift-multus/core/configmaps/cni-copy-resources.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992243227Z namespaces/openshift-multus/core/configmaps/default-cni-sysctl-allowlist.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992347833Z namespaces/openshift-multus/core/configmaps/multus-daemon-config.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99245304Z namespaces/openshift-multus/core/configmaps/whereabouts-flatfile-config.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992545455Z namespaces/openshift-multus/core/serviceaccounts/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992592878Z namespaces/openshift-multus/core/serviceaccounts/metrics-daemon-sa.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992684704Z namespaces/openshift-multus/core/serviceaccounts/multus-ac.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992763639Z namespaces/openshift-multus/core/serviceaccounts/multus-ancillary-tools.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992845714Z namespaces/openshift-multus/core/serviceaccounts/multus.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992897457Z namespaces/openshift-multus/core/services/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.992948511Z namespaces/openshift-multus/core/services/network-metrics-service.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993004104Z namespaces/openshift-multus/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993086629Z namespaces/openshift-multus/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993138432Z namespaces/openshift-multus/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993197956Z namespaces/openshift-multus/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993242099Z namespaces/openshift-multus/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993286141Z namespaces/openshift-multus/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993364146Z namespaces/openshift-multus/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993442121Z namespaces/openshift-multus/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993496024Z namespaces/openshift-multus/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993534937Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99358768Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993628662Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors/monitor-network.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993679016Z namespaces/openshift-multus/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993721178Z namespaces/openshift-multus/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993761451Z namespaces/openshift-multus/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993770981Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993829195Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.993979124Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.994182367Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.994322875Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.994514797Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.994702209Z namespaces/openshift-multus/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99471412Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.994751452Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/multus-additional-cni-plugins-4flfv.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.994894411Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/bond-cni-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.994907542Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.994912632Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.994925053Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995034649Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995102884Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995136176Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995158707Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/cni-plugins/cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995176988Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995232392Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995337498Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995397252Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995444595Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/egress-router-binary-copy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995458036Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995463376Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995485287Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995581423Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995643077Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99569029Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995702161Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995712761Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995760754Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995837089Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995905323Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995937985Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/routeoverride-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995949936Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995961467Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.995971297Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996076754Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996183571Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996226293Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni-bincopy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996236944Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996256675Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996290727Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996391203Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996459517Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99650249Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996512471Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996516991Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996543463Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996632148Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996716324Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-4flfv/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996736595Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996777747Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/multus-additional-cni-plugins-5bskg.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996905675Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/bond-cni-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996915346Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996919736Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.996951548Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997044884Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997113938Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997171462Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997192183Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/cni-plugins/cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997200174Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997218024Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99730656Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997374014Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997416817Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/egress-router-binary-copy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997427868Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997432268Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997452109Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997546095Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997612759Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997649341Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997656532Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997661172Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997691384Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997765838Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997841303Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997869885Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/routeoverride-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997876945Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997881265Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997910997Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.997999323Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998066487Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99811173Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni-bincopy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99811946Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998146642Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99827473Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998357815Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998430739Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998460151Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998465952Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998472202Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998519935Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998628822Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998664774Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-5bskg/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998720187Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998745229Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/multus-additional-cni-plugins-pgzsj.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998894148Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/bond-cni-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998907669Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998931681Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.998965503Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999113382Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999174605Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999189317Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999194457Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/cni-plugins/cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999199187Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99924381Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999339556Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99940902Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999434392Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/egress-router-binary-copy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999440032Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999444162Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999495015Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.99957704Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999655205Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999676036Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999687627Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999703938Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999740811Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999810195Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999880959Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999912161Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/routeoverride-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999918242Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999928382Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:10.999967945Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00005405Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000151016Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000205929Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni-bincopy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00021917Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00022403Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000301945Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000398311Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000466235Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000485766Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000502127Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000511208Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000588673Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000676668Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000738562Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-pgzsj/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000781055Z namespaces/openshift-multus/pods/multus-kjh44/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000851349Z namespaces/openshift-multus/pods/multus-kjh44/multus-kjh44.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000951015Z namespaces/openshift-multus/pods/multus-kjh44/kube-multus/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000958696Z namespaces/openshift-multus/pods/multus-kjh44/kube-multus/kube-multus/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000963136Z namespaces/openshift-multus/pods/multus-kjh44/kube-multus/kube-multus/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.000995568Z namespaces/openshift-multus/pods/multus-kjh44/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.001700082Z namespaces/openshift-multus/pods/multus-kjh44/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.001768586Z namespaces/openshift-multus/pods/multus-kjh44/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.001813999Z namespaces/openshift-multus/pods/multus-l8rvw/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.001854791Z namespaces/openshift-multus/pods/multus-l8rvw/multus-l8rvw.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.001949217Z namespaces/openshift-multus/pods/multus-l8rvw/kube-multus/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.001959878Z namespaces/openshift-multus/pods/multus-l8rvw/kube-multus/kube-multus/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.001964258Z namespaces/openshift-multus/pods/multus-l8rvw/kube-multus/kube-multus/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00199835Z namespaces/openshift-multus/pods/multus-l8rvw/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.002886885Z namespaces/openshift-multus/pods/multus-l8rvw/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00297288Z namespaces/openshift-multus/pods/multus-l8rvw/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.003013803Z namespaces/openshift-multus/pods/multus-zbvvt/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.003050425Z namespaces/openshift-multus/pods/multus-zbvvt/multus-zbvvt.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.003147211Z namespaces/openshift-multus/pods/multus-zbvvt/kube-multus/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.003173363Z namespaces/openshift-multus/pods/multus-zbvvt/kube-multus/kube-multus/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.003183864Z namespaces/openshift-multus/pods/multus-zbvvt/kube-multus/kube-multus/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.003240997Z namespaces/openshift-multus/pods/multus-zbvvt/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004005934Z namespaces/openshift-multus/pods/multus-zbvvt/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004077069Z namespaces/openshift-multus/pods/multus-zbvvt/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004141843Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004214447Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/network-metrics-daemon-5dnnt.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004296112Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004306893Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004311603Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004349305Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004452562Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004520756Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004539667Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/network-metrics-daemon/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004554468Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004565079Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004606001Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004880558Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004927401Z namespaces/openshift-multus/pods/network-metrics-daemon-5dnnt/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004932962Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004962254Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/network-metrics-daemon-d5x6f.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004977555Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004983425Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.004996796Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005008556Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005101102Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005194328Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00522342Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/network-metrics-daemon/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00523523Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005243301Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005285554Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005490556Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00555642Z namespaces/openshift-multus/pods/network-metrics-daemon-d5x6f/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005590542Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005622804Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/network-metrics-daemon-tgz85.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00571325Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00572127Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005725851Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005754783Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005848148Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005914972Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005962425Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/network-metrics-daemon/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005973876Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.005980076Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006015759Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006288265Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00636117Z namespaces/openshift-multus/pods/network-metrics-daemon-tgz85/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006392792Z namespaces/openshift-multus/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006439455Z namespaces/openshift-multus/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006493618Z namespaces/openshift-multus/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006502359Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006539801Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/multus-whereabouts.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006629746Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00668572Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006702551Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006811178Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/whereabouts-cni.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006838799Z namespaces/openshift-multus/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006889533Z namespaces/openshift-multus/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006932185Z namespaces/openshift-must-gather-kt8tj/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006940006Z namespaces/openshift-must-gather-kt8tj/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006944076Z namespaces/openshift-must-gather-kt8tj/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.006993809Z namespaces/openshift-must-gather-kt8tj/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.007187551Z namespaces/openshift-must-gather-kt8tj/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.007387913Z namespaces/openshift-must-gather-kt8tj/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.007523882Z namespaces/openshift-must-gather-kt8tj/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.007722934Z namespaces/openshift-must-gather-kt8tj/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.007920046Z namespaces/openshift-network-console/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.007940918Z namespaces/openshift-network-console/openshift-network-console.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008027983Z namespaces/openshift-network-console/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008048534Z namespaces/openshift-network-console/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008121979Z namespaces/openshift-network-console/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008374514Z namespaces/openshift-network-console/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008535664Z namespaces/openshift-network-console/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008598668Z namespaces/openshift-network-console/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008704125Z namespaces/openshift-network-console/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008755078Z namespaces/openshift-network-console/apps/deployments/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00879168Z namespaces/openshift-network-console/apps/deployments/networking-console-plugin.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008867545Z namespaces/openshift-network-console/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008920328Z namespaces/openshift-network-console/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008961461Z namespaces/openshift-network-console/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.008994993Z namespaces/openshift-network-console/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.009092879Z namespaces/openshift-network-console/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.00911333Z namespaces/openshift-network-console/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.009202876Z namespaces/openshift-network-console/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.009285391Z namespaces/openshift-network-console/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.009328833Z namespaces/openshift-network-console/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.009374736Z namespaces/openshift-network-console/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.009488823Z namespaces/openshift-network-console/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.009575309Z namespaces/openshift-network-console/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.009694536Z namespaces/openshift-network-console/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.009767031Z namespaces/openshift-network-console/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.0099158Z namespaces/openshift-network-console/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010046838Z namespaces/openshift-network-console/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010195537Z namespaces/openshift-network-console/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010249641Z namespaces/openshift-network-console/core/configmaps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010295453Z namespaces/openshift-network-console/core/configmaps/networking-console-plugin.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010346447Z namespaces/openshift-network-console/core/services/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010397239Z namespaces/openshift-network-console/core/services/networking-console-plugin.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010447143Z namespaces/openshift-network-console/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010480125Z namespaces/openshift-network-console/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010544189Z namespaces/openshift-network-console/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010592802Z namespaces/openshift-network-console/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010636545Z namespaces/openshift-network-console/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010679457Z namespaces/openshift-network-console/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010759992Z namespaces/openshift-network-console/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010848607Z namespaces/openshift-network-console/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01088385Z namespaces/openshift-network-console/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010910961Z namespaces/openshift-network-console/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.010973865Z namespaces/openshift-network-console/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.011000547Z namespaces/openshift-network-console/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01105286Z namespaces/openshift-network-console/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.011062461Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.011095283Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.011281024Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.011467706Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.011606484Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.011810527Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.011994348Z namespaces/openshift-network-console/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012003489Z namespaces/openshift-network-console/pods/networking-console-plugin-6cf856758c-d57tx/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012030801Z namespaces/openshift-network-console/pods/networking-console-plugin-6cf856758c-d57tx/networking-console-plugin-6cf856758c-d57tx.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012119486Z namespaces/openshift-network-console/pods/networking-console-plugin-6cf856758c-d57tx/networking-console-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012151278Z namespaces/openshift-network-console/pods/networking-console-plugin-6cf856758c-d57tx/networking-console-plugin/networking-console-plugin/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012155848Z namespaces/openshift-network-console/pods/networking-console-plugin-6cf856758c-d57tx/networking-console-plugin/networking-console-plugin/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012217822Z namespaces/openshift-network-console/pods/networking-console-plugin-6cf856758c-d57tx/networking-console-plugin/networking-console-plugin/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.0123413Z namespaces/openshift-network-console/pods/networking-console-plugin-6cf856758c-d57tx/networking-console-plugin/networking-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012412574Z namespaces/openshift-network-console/pods/networking-console-plugin-6cf856758c-d57tx/networking-console-plugin/networking-console-plugin/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012449186Z namespaces/openshift-network-console/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012495039Z namespaces/openshift-network-console/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012550903Z namespaces/openshift-network-console/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012587095Z namespaces/openshift-network-console/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012640588Z namespaces/openshift-network-diagnostics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012686171Z namespaces/openshift-network-diagnostics/openshift-network-diagnostics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012739764Z namespaces/openshift-network-diagnostics/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012781377Z namespaces/openshift-network-diagnostics/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012839071Z namespaces/openshift-network-diagnostics/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.012871853Z namespaces/openshift-network-diagnostics/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01298698Z namespaces/openshift-network-diagnostics/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013091026Z namespaces/openshift-network-diagnostics/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013207544Z namespaces/openshift-network-diagnostics/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013247436Z namespaces/openshift-network-diagnostics/apps/daemonsets/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013293459Z namespaces/openshift-network-diagnostics/apps/daemonsets/network-check-target.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013354102Z namespaces/openshift-network-diagnostics/apps/deployments/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013402845Z namespaces/openshift-network-diagnostics/apps/deployments/network-check-source.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01347742Z namespaces/openshift-network-diagnostics/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013511812Z namespaces/openshift-network-diagnostics/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013570846Z namespaces/openshift-network-diagnostics/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013603288Z namespaces/openshift-network-diagnostics/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013704724Z namespaces/openshift-network-diagnostics/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013744117Z namespaces/openshift-network-diagnostics/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013768268Z namespaces/openshift-network-diagnostics/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013865944Z namespaces/openshift-network-diagnostics/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013899206Z namespaces/openshift-network-diagnostics/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.013938459Z namespaces/openshift-network-diagnostics/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.014050396Z namespaces/openshift-network-diagnostics/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.014310752Z namespaces/openshift-network-diagnostics/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.014537226Z namespaces/openshift-network-diagnostics/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01460292Z namespaces/openshift-network-diagnostics/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.014810223Z namespaces/openshift-network-diagnostics/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.014885997Z namespaces/openshift-network-diagnostics/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015004224Z namespaces/openshift-network-diagnostics/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015061238Z namespaces/openshift-network-diagnostics/core/serviceaccounts/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015105211Z namespaces/openshift-network-diagnostics/core/serviceaccounts/network-diagnostics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015168805Z namespaces/openshift-network-diagnostics/core/services/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015217598Z namespaces/openshift-network-diagnostics/core/services/network-check-source.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015305293Z namespaces/openshift-network-diagnostics/core/services/network-check-target.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015385488Z namespaces/openshift-network-diagnostics/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015434691Z namespaces/openshift-network-diagnostics/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015449532Z namespaces/openshift-network-diagnostics/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015509096Z namespaces/openshift-network-diagnostics/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015532597Z namespaces/openshift-network-diagnostics/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015590571Z namespaces/openshift-network-diagnostics/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015682426Z namespaces/openshift-network-diagnostics/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015767392Z namespaces/openshift-network-diagnostics/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015790163Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015848297Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01589526Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015934882Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors/network-check-source.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.015974975Z namespaces/openshift-network-diagnostics/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.016015357Z namespaces/openshift-network-diagnostics/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01605436Z namespaces/openshift-network-diagnostics/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01606484Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.016117953Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.016349748Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01654979Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.016821807Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01703627Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017259294Z namespaces/openshift-network-diagnostics/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017273355Z namespaces/openshift-network-diagnostics/pods/network-check-source-869dcfcdcd-p8b85/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017316367Z namespaces/openshift-network-diagnostics/pods/network-check-source-869dcfcdcd-p8b85/network-check-source-869dcfcdcd-p8b85.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017398693Z namespaces/openshift-network-diagnostics/pods/network-check-source-869dcfcdcd-p8b85/check-endpoints/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017409953Z namespaces/openshift-network-diagnostics/pods/network-check-source-869dcfcdcd-p8b85/check-endpoints/check-endpoints/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017414584Z namespaces/openshift-network-diagnostics/pods/network-check-source-869dcfcdcd-p8b85/check-endpoints/check-endpoints/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017440765Z namespaces/openshift-network-diagnostics/pods/network-check-source-869dcfcdcd-p8b85/check-endpoints/check-endpoints/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017556022Z namespaces/openshift-network-diagnostics/pods/network-check-source-869dcfcdcd-p8b85/check-endpoints/check-endpoints/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017625857Z namespaces/openshift-network-diagnostics/pods/network-check-source-869dcfcdcd-p8b85/check-endpoints/check-endpoints/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01767204Z namespaces/openshift-network-diagnostics/pods/network-check-target-7xlv6/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017708772Z namespaces/openshift-network-diagnostics/pods/network-check-target-7xlv6/network-check-target-7xlv6.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017794007Z namespaces/openshift-network-diagnostics/pods/network-check-target-7xlv6/network-check-target-container/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017804648Z namespaces/openshift-network-diagnostics/pods/network-check-target-7xlv6/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017809258Z namespaces/openshift-network-diagnostics/pods/network-check-target-7xlv6/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017818899Z namespaces/openshift-network-diagnostics/pods/network-check-target-7xlv6/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.017928605Z namespaces/openshift-network-diagnostics/pods/network-check-target-7xlv6/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01799672Z namespaces/openshift-network-diagnostics/pods/network-check-target-7xlv6/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018032582Z namespaces/openshift-network-diagnostics/pods/network-check-target-9tvfn/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018071534Z namespaces/openshift-network-diagnostics/pods/network-check-target-9tvfn/network-check-target-9tvfn.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018145239Z namespaces/openshift-network-diagnostics/pods/network-check-target-9tvfn/network-check-target-container/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01816898Z namespaces/openshift-network-diagnostics/pods/network-check-target-9tvfn/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01817368Z namespaces/openshift-network-diagnostics/pods/network-check-target-9tvfn/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018207803Z namespaces/openshift-network-diagnostics/pods/network-check-target-9tvfn/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018301859Z namespaces/openshift-network-diagnostics/pods/network-check-target-9tvfn/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018372643Z namespaces/openshift-network-diagnostics/pods/network-check-target-9tvfn/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018406985Z namespaces/openshift-network-diagnostics/pods/network-check-target-jxtmc/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018449698Z namespaces/openshift-network-diagnostics/pods/network-check-target-jxtmc/network-check-target-jxtmc.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018521842Z namespaces/openshift-network-diagnostics/pods/network-check-target-jxtmc/network-check-target-container/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018529693Z namespaces/openshift-network-diagnostics/pods/network-check-target-jxtmc/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018536203Z namespaces/openshift-network-diagnostics/pods/network-check-target-jxtmc/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018569545Z namespaces/openshift-network-diagnostics/pods/network-check-target-jxtmc/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018661381Z namespaces/openshift-network-diagnostics/pods/network-check-target-jxtmc/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018727585Z namespaces/openshift-network-diagnostics/pods/network-check-target-jxtmc/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018763047Z namespaces/openshift-network-diagnostics/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018802759Z namespaces/openshift-network-diagnostics/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018843712Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018853463Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.018920257Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/network-diagnostics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019004972Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019047555Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019089097Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/network-diagnostics.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019201614Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019241977Z namespaces/openshift-network-diagnostics/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019282179Z namespaces/openshift-network-diagnostics/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019323692Z namespaces/openshift-network-node-identity/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019381995Z namespaces/openshift-network-node-identity/openshift-network-node-identity.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019437339Z namespaces/openshift-network-node-identity/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019480141Z namespaces/openshift-network-node-identity/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019519764Z namespaces/openshift-network-node-identity/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019642742Z namespaces/openshift-network-node-identity/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019724927Z namespaces/openshift-network-node-identity/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019812662Z namespaces/openshift-network-node-identity/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019894457Z namespaces/openshift-network-node-identity/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.01993475Z namespaces/openshift-network-node-identity/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.019975402Z namespaces/openshift-network-node-identity/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020017035Z namespaces/openshift-network-node-identity/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020057937Z namespaces/openshift-network-node-identity/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020408549Z namespaces/openshift-network-node-identity/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020470463Z namespaces/openshift-network-node-identity/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020521346Z namespaces/openshift-network-node-identity/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020604011Z namespaces/openshift-network-node-identity/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020645084Z namespaces/openshift-network-node-identity/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020651134Z namespaces/openshift-network-node-identity/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020678555Z namespaces/openshift-network-node-identity/coordination.k8s.io/leases/ovnkube-identity.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020739229Z namespaces/openshift-network-node-identity/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020770691Z namespaces/openshift-network-node-identity/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020880858Z namespaces/openshift-network-node-identity/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.020963273Z namespaces/openshift-network-node-identity/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021047948Z namespaces/openshift-network-node-identity/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021151235Z namespaces/openshift-network-node-identity/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021258922Z namespaces/openshift-network-node-identity/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021412201Z namespaces/openshift-network-node-identity/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021521098Z namespaces/openshift-network-node-identity/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021568701Z namespaces/openshift-network-node-identity/core/serviceaccounts/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021606173Z namespaces/openshift-network-node-identity/core/serviceaccounts/network-node-identity.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021660946Z namespaces/openshift-network-node-identity/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021697619Z namespaces/openshift-network-node-identity/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021745752Z namespaces/openshift-network-node-identity/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021782294Z namespaces/openshift-network-node-identity/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021826887Z namespaces/openshift-network-node-identity/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02187312Z namespaces/openshift-network-node-identity/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.021968495Z namespaces/openshift-network-node-identity/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022035979Z namespaces/openshift-network-node-identity/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022075332Z namespaces/openshift-network-node-identity/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022114084Z namespaces/openshift-network-node-identity/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022291615Z namespaces/openshift-network-node-identity/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022327428Z namespaces/openshift-network-node-identity/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022388121Z namespaces/openshift-network-node-identity/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022399702Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022418143Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022574273Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022751274Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.022888182Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023090065Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023307528Z namespaces/openshift-network-node-identity/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023357171Z namespaces/openshift-network-node-identity/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023397474Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023408435Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023443626Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/rolebindings/network-node-identity-leases.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02350239Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023535262Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/roles/network-node-identity-leases.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023590476Z namespaces/openshift-network-node-identity/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023629568Z namespaces/openshift-network-node-identity/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023695612Z namespaces/openshift-network-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023745365Z namespaces/openshift-network-operator/openshift-network-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023811149Z namespaces/openshift-network-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023839881Z namespaces/openshift-network-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023893714Z namespaces/openshift-network-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.023928687Z namespaces/openshift-network-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.024036713Z namespaces/openshift-network-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02413552Z namespaces/openshift-network-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.024258357Z namespaces/openshift-network-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02446659Z namespaces/openshift-network-operator/apps/daemonsets/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.024507452Z namespaces/openshift-network-operator/apps/daemonsets/iptables-alerter.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.024574436Z namespaces/openshift-network-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.024620289Z namespaces/openshift-network-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.024670393Z namespaces/openshift-network-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.024712765Z namespaces/openshift-network-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02479139Z namespaces/openshift-network-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.024993762Z namespaces/openshift-network-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.025030745Z namespaces/openshift-network-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02511243Z namespaces/openshift-network-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.025184414Z namespaces/openshift-network-operator/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.025200965Z namespaces/openshift-network-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.025222887Z namespaces/openshift-network-operator/coordination.k8s.io/leases/network-operator-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.025288541Z namespaces/openshift-network-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.025308492Z namespaces/openshift-network-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02544244Z namespaces/openshift-network-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.025525805Z namespaces/openshift-network-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.025679305Z namespaces/openshift-network-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02575857Z namespaces/openshift-network-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.025931671Z namespaces/openshift-network-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026016666Z namespaces/openshift-network-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026163605Z namespaces/openshift-network-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026223349Z namespaces/openshift-network-operator/core/configmaps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026272882Z namespaces/openshift-network-operator/core/configmaps/applied-cluster.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026356597Z namespaces/openshift-network-operator/core/configmaps/iptables-alerter-script.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02641791Z namespaces/openshift-network-operator/core/serviceaccounts/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026451213Z namespaces/openshift-network-operator/core/serviceaccounts/iptables-alerter.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026508176Z namespaces/openshift-network-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026537408Z namespaces/openshift-network-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026595521Z namespaces/openshift-network-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026625834Z namespaces/openshift-network-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026678707Z namespaces/openshift-network-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02673307Z namespaces/openshift-network-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026805905Z namespaces/openshift-network-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02688763Z namespaces/openshift-network-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026938313Z namespaces/openshift-network-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.026956634Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027016887Z namespaces/openshift-network-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027069581Z namespaces/openshift-network-operator/monitoring.coreos.com/prometheusrules/openshift-network-operator-ipsec-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027135425Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027158736Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors/network-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027236771Z namespaces/openshift-network-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027272854Z namespaces/openshift-network-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027316746Z namespaces/openshift-network-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027331077Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027351758Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027507638Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027683059Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.027821627Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028018409Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028274736Z namespaces/openshift-network-operator/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028286466Z namespaces/openshift-network-operator/pods/iptables-alerter-7xfrf/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028325428Z namespaces/openshift-network-operator/pods/iptables-alerter-7xfrf/iptables-alerter-7xfrf.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028398873Z namespaces/openshift-network-operator/pods/iptables-alerter-7xfrf/iptables-alerter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028409614Z namespaces/openshift-network-operator/pods/iptables-alerter-7xfrf/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028414214Z namespaces/openshift-network-operator/pods/iptables-alerter-7xfrf/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028456607Z namespaces/openshift-network-operator/pods/iptables-alerter-7xfrf/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028552113Z namespaces/openshift-network-operator/pods/iptables-alerter-7xfrf/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028620987Z namespaces/openshift-network-operator/pods/iptables-alerter-7xfrf/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028660119Z namespaces/openshift-network-operator/pods/iptables-alerter-df5s8/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02868035Z namespaces/openshift-network-operator/pods/iptables-alerter-df5s8/iptables-alerter-df5s8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028762835Z namespaces/openshift-network-operator/pods/iptables-alerter-df5s8/iptables-alerter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028773756Z namespaces/openshift-network-operator/pods/iptables-alerter-df5s8/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028779016Z namespaces/openshift-network-operator/pods/iptables-alerter-df5s8/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028807798Z namespaces/openshift-network-operator/pods/iptables-alerter-df5s8/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028896654Z namespaces/openshift-network-operator/pods/iptables-alerter-df5s8/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.028964148Z namespaces/openshift-network-operator/pods/iptables-alerter-df5s8/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.02899449Z namespaces/openshift-network-operator/pods/iptables-alerter-shrmd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029039623Z namespaces/openshift-network-operator/pods/iptables-alerter-shrmd/iptables-alerter-shrmd.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029109397Z namespaces/openshift-network-operator/pods/iptables-alerter-shrmd/iptables-alerter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029118377Z namespaces/openshift-network-operator/pods/iptables-alerter-shrmd/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029141879Z namespaces/openshift-network-operator/pods/iptables-alerter-shrmd/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029189062Z namespaces/openshift-network-operator/pods/iptables-alerter-shrmd/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029300409Z namespaces/openshift-network-operator/pods/iptables-alerter-shrmd/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029368843Z namespaces/openshift-network-operator/pods/iptables-alerter-shrmd/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029382794Z namespaces/openshift-network-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029437887Z namespaces/openshift-network-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029496741Z namespaces/openshift-network-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029515512Z namespaces/openshift-network-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029574076Z namespaces/openshift-node/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029585986Z namespaces/openshift-node/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029591447Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029618208Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029768568Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.029939638Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.030084527Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.030409628Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.030678224Z namespaces/openshift-operator-lifecycle-manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.030720677Z namespaces/openshift-operator-lifecycle-manager/openshift-operator-lifecycle-manager.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.030792071Z namespaces/openshift-operator-lifecycle-manager/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.030838084Z namespaces/openshift-operator-lifecycle-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.030881617Z namespaces/openshift-operator-lifecycle-manager/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.030922649Z namespaces/openshift-operator-lifecycle-manager/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031006594Z namespaces/openshift-operator-lifecycle-manager/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.03109332Z namespaces/openshift-operator-lifecycle-manager/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031206307Z namespaces/openshift-operator-lifecycle-manager/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.03126476Z namespaces/openshift-operator-lifecycle-manager/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031300903Z namespaces/openshift-operator-lifecycle-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031354576Z namespaces/openshift-operator-lifecycle-manager/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031388298Z namespaces/openshift-operator-lifecycle-manager/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031481634Z namespaces/openshift-operator-lifecycle-manager/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031514576Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031569139Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031650824Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031684286Z namespaces/openshift-operator-lifecycle-manager/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031722759Z namespaces/openshift-operator-lifecycle-manager/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031829215Z namespaces/openshift-operator-lifecycle-manager/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.031920081Z namespaces/openshift-operator-lifecycle-manager/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032010396Z namespaces/openshift-operator-lifecycle-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032087761Z namespaces/openshift-operator-lifecycle-manager/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032206779Z namespaces/openshift-operator-lifecycle-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032360848Z namespaces/openshift-operator-lifecycle-manager/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032564791Z namespaces/openshift-operator-lifecycle-manager/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032621634Z namespaces/openshift-operator-lifecycle-manager/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032674137Z namespaces/openshift-operator-lifecycle-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032726901Z namespaces/openshift-operator-lifecycle-manager/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032759363Z namespaces/openshift-operator-lifecycle-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032814976Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032865799Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.032943914Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033022939Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033072182Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033114625Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033182389Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033225912Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/prometheusrules/olm-alert-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033290846Z namespaces/openshift-operator-lifecycle-manager/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033309277Z namespaces/openshift-operator-lifecycle-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033413703Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033425184Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033435544Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033594054Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033776656Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.033915644Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034110006Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034313999Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.03433275Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/operatorgroups/olm-operators.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034401444Z namespaces/openshift-operator-lifecycle-manager/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034451467Z namespaces/openshift-operator-lifecycle-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034505181Z namespaces/openshift-operator-lifecycle-manager/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034547113Z namespaces/openshift-operator-lifecycle-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034590266Z namespaces/openshift-operators/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034598256Z namespaces/openshift-operators/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034602507Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034637529Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034785948Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.034967179Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.035112218Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.035365844Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.035642941Z namespaces/openshift-operators/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.035663042Z namespaces/openshift-operators/operators.coreos.com/operatorgroups/global-operators.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.035727286Z namespaces/openshift-ovn-kubernetes/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.035762848Z namespaces/openshift-ovn-kubernetes/openshift-ovn-kubernetes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.035836943Z namespaces/openshift-ovn-kubernetes/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.035869325Z namespaces/openshift-ovn-kubernetes/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.035924398Z namespaces/openshift-ovn-kubernetes/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.03594936Z namespaces/openshift-ovn-kubernetes/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.036253579Z namespaces/openshift-ovn-kubernetes/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.036354625Z namespaces/openshift-ovn-kubernetes/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.0364348Z namespaces/openshift-ovn-kubernetes/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.036482293Z namespaces/openshift-ovn-kubernetes/apps/daemonsets/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.036517345Z namespaces/openshift-ovn-kubernetes/apps/daemonsets/ovnkube-node.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.036655114Z namespaces/openshift-ovn-kubernetes/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.036688036Z namespaces/openshift-ovn-kubernetes/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.03675463Z namespaces/openshift-ovn-kubernetes/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.036772931Z namespaces/openshift-ovn-kubernetes/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.036865307Z namespaces/openshift-ovn-kubernetes/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.036908839Z namespaces/openshift-ovn-kubernetes/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.036947362Z namespaces/openshift-ovn-kubernetes/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.037034847Z namespaces/openshift-ovn-kubernetes/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.03708051Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.037087561Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.037140044Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/leases/ovn-kubernetes-master.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.037200528Z namespaces/openshift-ovn-kubernetes/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.03724747Z namespaces/openshift-ovn-kubernetes/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.037446703Z namespaces/openshift-ovn-kubernetes/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.037530358Z namespaces/openshift-ovn-kubernetes/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.037839877Z namespaces/openshift-ovn-kubernetes/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.037967475Z namespaces/openshift-ovn-kubernetes/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.03853856Z namespaces/openshift-ovn-kubernetes/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.03870194Z namespaces/openshift-ovn-kubernetes/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.038916994Z namespaces/openshift-ovn-kubernetes/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.038955146Z namespaces/openshift-ovn-kubernetes/core/configmaps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039010049Z namespaces/openshift-ovn-kubernetes/core/configmaps/ovnkube-config.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039094294Z namespaces/openshift-ovn-kubernetes/core/configmaps/ovnkube-script-lib.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039238603Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039279256Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ovn-kubernetes-control-plane.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039367151Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ovn-kubernetes-node.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039426665Z namespaces/openshift-ovn-kubernetes/core/services/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039446346Z namespaces/openshift-ovn-kubernetes/core/services/ovn-kubernetes-node.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039515971Z namespaces/openshift-ovn-kubernetes/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039550263Z namespaces/openshift-ovn-kubernetes/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039655619Z namespaces/openshift-ovn-kubernetes/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039676461Z namespaces/openshift-ovn-kubernetes/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039738934Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039747895Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/network-attachment-definitions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039858722Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/network-attachment-definitions/default.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039918136Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.039936277Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040028723Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040138129Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040222964Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040503362Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040562625Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040626039Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/prometheusrules/networking-rules.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040715675Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040752437Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors/monitor-ovn-node.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040811171Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040822132Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040859994Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/ovn.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.040949429Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/signer.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.041010203Z namespaces/openshift-ovn-kubernetes/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.041025104Z namespaces/openshift-ovn-kubernetes/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.041085518Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.041092968Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.041140871Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.041327453Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.041516444Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.041658433Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.041866846Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042043787Z namespaces/openshift-ovn-kubernetes/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042053448Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042106621Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovnkube-node-2vqpm.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042317574Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-node/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042331825Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042336605Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042350406Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042480834Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042551318Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.04258299Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042587951Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042592881Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042633833Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042764432Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042838406Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042854247Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/nbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042858817Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/nbdb/nbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042865048Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/nbdb/nbdb/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.042910931Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/nbdb/nbdb/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043014007Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043078681Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043141865Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/northd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043154716Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/northd/northd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043159186Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/northd/northd/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043186938Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/northd/northd/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043300365Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043363769Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/northd/northd/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.04339219Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-acl-logging/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043399201Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043407801Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043448414Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043558801Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043574902Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043587223Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-controller/ovn-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043594613Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043645166Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043880441Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043941284Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043985277Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovnkube-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043993938Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.043999078Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.044009329Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.045585806Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.045634129Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.045664821Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/sbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.045669811Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/sbdb/sbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.045673921Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/sbdb/sbdb/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.045720464Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/sbdb/sbdb/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.045822201Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.045884245Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-2vqpm/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.045929217Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.045999202Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovnkube-node-vhthf.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046160432Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-node/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046248647Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046259128Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.04629519Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.04645614Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046512573Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046556996Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046565547Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046569877Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046597809Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046720616Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046787761Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046828933Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/nbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046834803Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/nbdb/nbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046841794Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/nbdb/nbdb/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046871635Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/nbdb/nbdb/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.046969702Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047037816Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047081408Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/northd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047088999Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/northd/northd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047093109Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/northd/northd/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047136992Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/northd/northd/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047247399Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047315983Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/northd/northd/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047354776Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-acl-logging/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047360106Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047364216Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047399238Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047488024Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-acl-logging/ovn-acl-logging/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047559118Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.04759385Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047599031Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-controller/ovn-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047603041Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047643653Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047931081Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.047997485Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.048035528Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovnkube-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.048040628Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.048044908Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.04807999Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.049970727Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050033341Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050071093Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/sbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050084244Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/sbdb/sbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050088755Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/sbdb/sbdb/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050118407Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/sbdb/sbdb/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050252655Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050310318Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vhthf/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050353941Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050421505Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovnkube-node-vm2hw.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050571555Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-node/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050579075Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050583375Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050616087Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050748075Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05081794Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050850232Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050857092Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050861122Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.050902985Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051029103Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051100007Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05115398Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/nbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051167221Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/nbdb/nbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051171461Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/nbdb/nbdb/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051203583Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/nbdb/nbdb/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051299909Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051368564Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051409236Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/northd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051416687Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/northd/northd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051420767Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/northd/northd/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051454719Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/northd/northd/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051558985Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05162717Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/northd/northd/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051668682Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-acl-logging/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051675883Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051679993Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051711415Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051806181Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051855734Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051867855Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-controller/ovn-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051871945Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.051916548Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.052665284Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.052740209Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05276015Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovnkube-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05276983Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.052780971Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.052815713Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.054561321Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.054633726Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.054646947Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/sbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.054657007Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/sbdb/sbdb/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.054663068Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/sbdb/sbdb/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.054721981Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/sbdb/sbdb/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.054805706Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.054889682Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-vm2hw/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.054904192Z namespaces/openshift-ovn-kubernetes/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.054960806Z namespaces/openshift-ovn-kubernetes/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055005739Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055016749Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055041381Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055148707Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/openshift-ovn-kubernetes-nodes-identity-limited.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055373071Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055427315Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055456126Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055559443Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/openshift-ovn-kubernetes-node-limited.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055636038Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055689851Z namespaces/openshift-ovn-kubernetes/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055718663Z namespaces/openshift-ovn-kubernetes/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055775916Z namespaces/openshift-route-controller-manager/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055784417Z namespaces/openshift-route-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055789297Z namespaces/openshift-route-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05583128Z namespaces/openshift-route-controller-manager/coordination.k8s.io/leases/openshift-route-controllers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055882423Z namespaces/openshift-route-controller-manager/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055893454Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.055904574Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.0561633Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.056398105Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.056545554Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.056758427Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.056981711Z namespaces/openshift-service-ca-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057024003Z namespaces/openshift-service-ca-operator/openshift-service-ca-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057086187Z namespaces/openshift-service-ca-operator/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057120129Z namespaces/openshift-service-ca-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057210655Z namespaces/openshift-service-ca-operator/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057270709Z namespaces/openshift-service-ca-operator/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057350784Z namespaces/openshift-service-ca-operator/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057464021Z namespaces/openshift-service-ca-operator/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057560107Z namespaces/openshift-service-ca-operator/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057591529Z namespaces/openshift-service-ca-operator/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057643872Z namespaces/openshift-service-ca-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057742378Z namespaces/openshift-service-ca-operator/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05778011Z namespaces/openshift-service-ca-operator/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057866106Z namespaces/openshift-service-ca-operator/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057910008Z namespaces/openshift-service-ca-operator/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.057951051Z namespaces/openshift-service-ca-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.058053037Z namespaces/openshift-service-ca-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05808936Z namespaces/openshift-service-ca-operator/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05809569Z namespaces/openshift-service-ca-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.058148783Z namespaces/openshift-service-ca-operator/coordination.k8s.io/leases/service-ca-operator-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.058234389Z namespaces/openshift-service-ca-operator/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.058283121Z namespaces/openshift-service-ca-operator/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.0584193Z namespaces/openshift-service-ca-operator/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.058489124Z namespaces/openshift-service-ca-operator/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.058684656Z namespaces/openshift-service-ca-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05875307Z namespaces/openshift-service-ca-operator/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.058870638Z namespaces/openshift-service-ca-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059002256Z namespaces/openshift-service-ca-operator/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059221369Z namespaces/openshift-service-ca-operator/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059287654Z namespaces/openshift-service-ca-operator/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059345137Z namespaces/openshift-service-ca-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05938963Z namespaces/openshift-service-ca-operator/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059431152Z namespaces/openshift-service-ca-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059484116Z namespaces/openshift-service-ca-operator/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059526948Z namespaces/openshift-service-ca-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059607243Z namespaces/openshift-service-ca-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059687948Z namespaces/openshift-service-ca-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059731991Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059773433Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059827637Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.05988324Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors/service-ca-operator.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059928073Z namespaces/openshift-service-ca-operator/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.059974046Z namespaces/openshift-service-ca-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.060016329Z namespaces/openshift-service-ca-operator/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.060024129Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.060057811Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.060241082Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.060567703Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.060705191Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.060906274Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061093245Z namespaces/openshift-service-ca-operator/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061102016Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-7695fc6c8c-l9v9d/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061141758Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-7695fc6c8c-l9v9d/service-ca-operator-7695fc6c8c-l9v9d.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061242244Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-7695fc6c8c-l9v9d/service-ca-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061250055Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-7695fc6c8c-l9v9d/service-ca-operator/service-ca-operator/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061253955Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-7695fc6c8c-l9v9d/service-ca-operator/service-ca-operator/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061290547Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-7695fc6c8c-l9v9d/service-ca-operator/service-ca-operator/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061445117Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-7695fc6c8c-l9v9d/service-ca-operator/service-ca-operator/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061593176Z namespaces/openshift-service-ca-operator/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061669931Z namespaces/openshift-service-ca-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061722954Z namespaces/openshift-service-ca-operator/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061762627Z namespaces/openshift-service-ca-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.06181189Z namespaces/openshift-service-ca/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061856453Z namespaces/openshift-service-ca/openshift-service-ca.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061912196Z namespaces/openshift-service-ca/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.061951938Z namespaces/openshift-service-ca/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062008612Z namespaces/openshift-service-ca/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062042634Z namespaces/openshift-service-ca/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062147431Z namespaces/openshift-service-ca/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062325422Z namespaces/openshift-service-ca/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062413207Z namespaces/openshift-service-ca/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.06246075Z namespaces/openshift-service-ca/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062493092Z namespaces/openshift-service-ca/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062552265Z namespaces/openshift-service-ca/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062569937Z namespaces/openshift-service-ca/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062670893Z namespaces/openshift-service-ca/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062715155Z namespaces/openshift-service-ca/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062757888Z namespaces/openshift-service-ca/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062842623Z namespaces/openshift-service-ca/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062873315Z namespaces/openshift-service-ca/coordination.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062882346Z namespaces/openshift-service-ca/coordination.k8s.io/leases/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062934559Z namespaces/openshift-service-ca/coordination.k8s.io/leases/service-ca-controller-lock.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.062984312Z namespaces/openshift-service-ca/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.063027215Z namespaces/openshift-service-ca/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.063158543Z namespaces/openshift-service-ca/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.063254169Z namespaces/openshift-service-ca/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.063390027Z namespaces/openshift-service-ca/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.063458261Z namespaces/openshift-service-ca/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.063571968Z namespaces/openshift-service-ca/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.063698706Z namespaces/openshift-service-ca/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.063875307Z namespaces/openshift-service-ca/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.06391252Z namespaces/openshift-service-ca/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.063964893Z namespaces/openshift-service-ca/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064002715Z namespaces/openshift-service-ca/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064053188Z namespaces/openshift-service-ca/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.0640884Z namespaces/openshift-service-ca/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064152315Z namespaces/openshift-service-ca/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064341016Z namespaces/openshift-service-ca/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064422971Z namespaces/openshift-service-ca/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064471124Z namespaces/openshift-service-ca/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064518057Z namespaces/openshift-service-ca/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.0645659Z namespaces/openshift-service-ca/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064606643Z namespaces/openshift-service-ca/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064652215Z namespaces/openshift-service-ca/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064660586Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064700088Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.064846267Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.06505103Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.065226431Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.065435294Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.065625626Z namespaces/openshift-service-ca/pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.065633106Z namespaces/openshift-service-ca/pods/service-ca-79f68b9f4f-xm4rd/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.065664048Z namespaces/openshift-service-ca/pods/service-ca-79f68b9f4f-xm4rd/service-ca-79f68b9f4f-xm4rd.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.06585611Z namespaces/openshift-service-ca/pods/service-ca-79f68b9f4f-xm4rd/service-ca-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.06586374Z namespaces/openshift-service-ca/pods/service-ca-79f68b9f4f-xm4rd/service-ca-controller/service-ca-controller/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.065873901Z namespaces/openshift-service-ca/pods/service-ca-79f68b9f4f-xm4rd/service-ca-controller/service-ca-controller/logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.065896282Z namespaces/openshift-service-ca/pods/service-ca-79f68b9f4f-xm4rd/service-ca-controller/service-ca-controller/logs/current.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066113286Z namespaces/openshift-service-ca/pods/service-ca-79f68b9f4f-xm4rd/service-ca-controller/service-ca-controller/logs/previous.insecure.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066202751Z namespaces/openshift-service-ca/pods/service-ca-79f68b9f4f-xm4rd/service-ca-controller/service-ca-controller/logs/previous.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066224653Z namespaces/openshift-service-ca/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066277466Z namespaces/openshift-service-ca/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066326619Z namespaces/openshift-service-ca/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066359031Z namespaces/openshift-service-ca/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066415604Z namespaces/openshift-user-workload-monitoring/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066445496Z namespaces/openshift-user-workload-monitoring/openshift-user-workload-monitoring.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066517981Z namespaces/openshift-user-workload-monitoring/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066545683Z namespaces/openshift-user-workload-monitoring/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066605116Z namespaces/openshift-user-workload-monitoring/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066632508Z namespaces/openshift-user-workload-monitoring/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066727824Z namespaces/openshift-user-workload-monitoring/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066807459Z namespaces/openshift-user-workload-monitoring/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066893254Z namespaces/openshift-user-workload-monitoring/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066936057Z namespaces/openshift-user-workload-monitoring/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.066976249Z namespaces/openshift-user-workload-monitoring/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067014292Z namespaces/openshift-user-workload-monitoring/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067064205Z namespaces/openshift-user-workload-monitoring/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067169121Z namespaces/openshift-user-workload-monitoring/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067217044Z namespaces/openshift-user-workload-monitoring/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067257347Z namespaces/openshift-user-workload-monitoring/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067338621Z namespaces/openshift-user-workload-monitoring/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067391935Z namespaces/openshift-user-workload-monitoring/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067427557Z namespaces/openshift-user-workload-monitoring/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067531844Z namespaces/openshift-user-workload-monitoring/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067611278Z namespaces/openshift-user-workload-monitoring/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067692683Z namespaces/openshift-user-workload-monitoring/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067777189Z namespaces/openshift-user-workload-monitoring/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067859314Z namespaces/openshift-user-workload-monitoring/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.067989322Z namespaces/openshift-user-workload-monitoring/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068192474Z namespaces/openshift-user-workload-monitoring/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068247778Z namespaces/openshift-user-workload-monitoring/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068294321Z namespaces/openshift-user-workload-monitoring/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068329603Z namespaces/openshift-user-workload-monitoring/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068373916Z namespaces/openshift-user-workload-monitoring/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068413988Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.06844478Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068538966Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068614511Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068666924Z namespaces/openshift-user-workload-monitoring/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068704086Z namespaces/openshift-user-workload-monitoring/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068754139Z namespaces/openshift-user-workload-monitoring/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068795752Z namespaces/openshift-user-workload-monitoring/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068848745Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068859336Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.068871266Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.069042247Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.069268471Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.069444582Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.069646554Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.069827335Z namespaces/openshift-user-workload-monitoring/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.069868978Z namespaces/openshift-user-workload-monitoring/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.069919121Z namespaces/openshift-user-workload-monitoring/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.069961074Z namespaces/openshift-user-workload-monitoring/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070012317Z namespaces/openshift/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070054449Z namespaces/openshift/openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070117053Z namespaces/openshift/apps.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070257392Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070305875Z namespaces/openshift/apps/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070335547Z namespaces/openshift/apps/daemonsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070427072Z namespaces/openshift/apps/deployments.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.07054651Z namespaces/openshift/apps/replicasets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070624065Z namespaces/openshift/apps/statefulsets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070668378Z namespaces/openshift/autoscaling/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.07071363Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070760713Z namespaces/openshift/batch/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070788385Z namespaces/openshift/batch/cronjobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070886411Z namespaces/openshift/batch/jobs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070925794Z namespaces/openshift/build.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.070955425Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071047521Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071088553Z namespaces/openshift/core/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071140447Z namespaces/openshift/core/configmaps.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071285996Z namespaces/openshift/core/endpoints.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071363181Z namespaces/openshift/core/events.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071444455Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071527361Z namespaces/openshift/core/pods.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071603975Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071731243Z namespaces/openshift/core/secrets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071905884Z namespaces/openshift/core/services.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.071955657Z namespaces/openshift/discovery.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.07200086Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.072050213Z namespaces/openshift/image.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.072088765Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.073407007Z namespaces/openshift/image.openshift.io/imagestreams/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.073426328Z namespaces/openshift/image.openshift.io/imagestreams/cli-artifacts.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.073535755Z namespaces/openshift/image.openshift.io/imagestreams/cli.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.07361642Z namespaces/openshift/image.openshift.io/imagestreams/dotnet-runtime.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.073726647Z namespaces/openshift/image.openshift.io/imagestreams/dotnet.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.073833703Z namespaces/openshift/image.openshift.io/imagestreams/driver-toolkit.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.073919208Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-eap-openshift-java11.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.074079509Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-eap-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.074325774Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-java-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.074453201Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-java11-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.074547707Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-karaf-openshift-jdk11.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.074639243Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-karaf-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.074761211Z namespaces/openshift/image.openshift.io/imagestreams/golang.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.074865327Z namespaces/openshift/image.openshift.io/imagestreams/httpd.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.074971474Z namespaces/openshift/image.openshift.io/imagestreams/installer-artifacts.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.075055919Z namespaces/openshift/image.openshift.io/imagestreams/installer.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.075169736Z namespaces/openshift/image.openshift.io/imagestreams/java-runtime.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.075300874Z namespaces/openshift/image.openshift.io/imagestreams/java.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.075418631Z namespaces/openshift/image.openshift.io/imagestreams/jboss-datagrid73-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.075533288Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp3-openjdk11-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.075626224Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp3-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.075733121Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp4-openjdk11-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.075837527Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp4-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.075938133Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk11-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.076034789Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.076136936Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk8-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.076257653Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk8-runtime-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.076351999Z namespaces/openshift/image.openshift.io/imagestreams/jboss-webserver57-openjdk11-tomcat9-openshift-ubi8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.076458076Z namespaces/openshift/image.openshift.io/imagestreams/jboss-webserver57-openjdk8-tomcat9-openshift-ubi8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.076557012Z namespaces/openshift/image.openshift.io/imagestreams/jenkins-agent-base.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.076653138Z namespaces/openshift/image.openshift.io/imagestreams/jenkins.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.076765575Z namespaces/openshift/image.openshift.io/imagestreams/mariadb.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.076872991Z namespaces/openshift/image.openshift.io/imagestreams/must-gather.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.076959197Z namespaces/openshift/image.openshift.io/imagestreams/mysql.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.077072194Z namespaces/openshift/image.openshift.io/imagestreams/network-tools.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.07718209Z namespaces/openshift/image.openshift.io/imagestreams/nginx.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.077312389Z namespaces/openshift/image.openshift.io/imagestreams/nodejs.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.077447627Z namespaces/openshift/image.openshift.io/imagestreams/oauth-proxy.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.077536092Z namespaces/openshift/image.openshift.io/imagestreams/openjdk-11-rhel7.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.07765715Z namespaces/openshift/image.openshift.io/imagestreams/perl.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.077751676Z namespaces/openshift/image.openshift.io/imagestreams/php.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.077864063Z namespaces/openshift/image.openshift.io/imagestreams/postgresql.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.07799414Z namespaces/openshift/image.openshift.io/imagestreams/postgresql13-for-sso75-openshift-rhel8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.078084206Z namespaces/openshift/image.openshift.io/imagestreams/postgresql13-for-sso76-openshift-rhel8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.078201613Z namespaces/openshift/image.openshift.io/imagestreams/python.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.078340432Z namespaces/openshift/image.openshift.io/imagestreams/redhat-openjdk18-openshift.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.07847085Z namespaces/openshift/image.openshift.io/imagestreams/redis.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.078577337Z namespaces/openshift/image.openshift.io/imagestreams/ruby.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.078689814Z namespaces/openshift/image.openshift.io/imagestreams/sso75-openshift-rhel8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.07878655Z namespaces/openshift/image.openshift.io/imagestreams/sso76-openshift-rhel8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.078881206Z namespaces/openshift/image.openshift.io/imagestreams/tests.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.078974051Z namespaces/openshift/image.openshift.io/imagestreams/tools.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.079093809Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-11-runtime.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.079250278Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-11.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.079364175Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-17-runtime.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.079470572Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-17.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.079594499Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-21-runtime.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.079697456Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-21.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.079784751Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-8-runtime.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.079904249Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-8.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.079996835Z namespaces/openshift/k8s.ovn.org/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080041197Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080141083Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080236299Z namespaces/openshift/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080284092Z namespaces/openshift/monitoring.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080321844Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080375138Z namespaces/openshift/networking.k8s.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.08040678Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080458883Z namespaces/openshift/operators.coreos.com/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080465993Z namespaces/openshift/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080500526Z namespaces/openshift/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080652545Z namespaces/openshift/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080836547Z namespaces/openshift/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.080972975Z namespaces/openshift/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.081192318Z namespaces/openshift/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.081460925Z namespaces/openshift/policy/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.081499397Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.08154589Z namespaces/openshift/route.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.081591813Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.081638556Z namespaces/openshift/template.openshift.io/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.081646606Z namespaces/openshift/template.openshift.io/templates/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.081681179Z namespaces/openshift/template.openshift.io/templates/cache-service.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.081795036Z namespaces/openshift/template.openshift.io/templates/cakephp-mysql-example.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.081937564Z namespaces/openshift/template.openshift.io/templates/cakephp-mysql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.082047901Z namespaces/openshift/template.openshift.io/templates/dancer-mysql-example.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.082234553Z namespaces/openshift/template.openshift.io/templates/dancer-mysql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.082414724Z namespaces/openshift/template.openshift.io/templates/datagrid-service.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.0825192Z namespaces/openshift/template.openshift.io/templates/django-psql-example.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.082639418Z namespaces/openshift/template.openshift.io/templates/django-psql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.082754135Z namespaces/openshift/template.openshift.io/templates/eap-xp3-basic-s2i.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.082880913Z namespaces/openshift/template.openshift.io/templates/eap-xp4-basic-s2i.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.08299117Z namespaces/openshift/template.openshift.io/templates/eap74-basic-s2i.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.083102707Z namespaces/openshift/template.openshift.io/templates/eap74-https-s2i.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.083271637Z namespaces/openshift/template.openshift.io/templates/eap74-sso-s2i.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.083416976Z namespaces/openshift/template.openshift.io/templates/httpd-example.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.083520112Z namespaces/openshift/template.openshift.io/templates/jenkins-ephemeral-monitored.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.08364201Z namespaces/openshift/template.openshift.io/templates/jenkins-ephemeral.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.083748687Z namespaces/openshift/template.openshift.io/templates/jenkins-persistent-monitored.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.083864324Z namespaces/openshift/template.openshift.io/templates/jenkins-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.083975221Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk11-tomcat9-ubi8-basic-s2i.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.084081677Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk11-tomcat9-ubi8-https-s2i.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.084232877Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk8-tomcat9-ubi8-basic-s2i.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.084348804Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk8-tomcat9-ubi8-https-s2i.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.08445595Z namespaces/openshift/template.openshift.io/templates/mariadb-ephemeral.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.084565927Z namespaces/openshift/template.openshift.io/templates/mariadb-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.084669764Z namespaces/openshift/template.openshift.io/templates/mysql-ephemeral.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.08477647Z namespaces/openshift/template.openshift.io/templates/mysql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.084877986Z namespaces/openshift/template.openshift.io/templates/nginx-example.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.084982793Z namespaces/openshift/template.openshift.io/templates/nodejs-postgresql-example.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.08509747Z namespaces/openshift/template.openshift.io/templates/nodejs-postgresql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.085236048Z namespaces/openshift/template.openshift.io/templates/openjdk-web-basic-s2i.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.085341075Z namespaces/openshift/template.openshift.io/templates/postgresql-ephemeral.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.085446112Z namespaces/openshift/template.openshift.io/templates/postgresql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.085548518Z namespaces/openshift/template.openshift.io/templates/rails-pgsql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.085672866Z namespaces/openshift/template.openshift.io/templates/rails-postgresql-example.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.085790583Z namespaces/openshift/template.openshift.io/templates/react-web-app-example.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.085898989Z namespaces/openshift/template.openshift.io/templates/redis-ephemeral.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.086007016Z namespaces/openshift/template.openshift.io/templates/redis-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.086103242Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel-rest-3scale.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.086243261Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel-xml.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.086349837Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.086458354Z namespaces/openshift/template.openshift.io/templates/sso75-https.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.086585242Z namespaces/openshift/template.openshift.io/templates/sso75-ocp4-x509-https.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.086699069Z namespaces/openshift/template.openshift.io/templates/sso75-ocp4-x509-postgresql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.086819906Z namespaces/openshift/template.openshift.io/templates/sso75-postgresql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.086943914Z namespaces/openshift/template.openshift.io/templates/sso75-postgresql.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.087083783Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-https.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.087314827Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-postgresql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.087447205Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-postgresql.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.087582774Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-x509-https.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.087699821Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-x509-postgresql-persistent.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.08784853Z network_logs/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.087888743Z network_logs/cluster_scale [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.087975738Z network_logs/ippools.whereabouts.cni.cncf.io [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.088049173Z network_logs/multi-networkpolicy [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.088145908Z network_logs/net-attach-def [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.088275576Z network_logs/overlappingrangeipreservations.whereabouts.cni.cncf.io [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.088421796Z network_logs/ovn_kubernetes_top_pods [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.088514341Z network_logs/ovnk_database_store.tar.gz [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.091004645Z nodes/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.0910803Z nodes/debug [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.091192797Z nodes/ip-10-0-128-180.ec2.internal/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.091214148Z nodes/ip-10-0-128-180.ec2.internal/cpu_affinities.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.091538758Z nodes/ip-10-0-128-180.ec2.internal/dmesg [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.091820866Z nodes/ip-10-0-128-180.ec2.internal/ethtool_channels [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.091900351Z nodes/ip-10-0-128-180.ec2.internal/ethtool_features [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.091993196Z nodes/ip-10-0-128-180.ec2.internal/ip-10-0-128-180.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.092353129Z nodes/ip-10-0-128-180.ec2.internal/irq_affinities.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.092505668Z nodes/ip-10-0-128-180.ec2.internal/lscpu [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.092639226Z nodes/ip-10-0-128-180.ec2.internal/lspci [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.092733492Z nodes/ip-10-0-128-180.ec2.internal/podresources.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.09285189Z nodes/ip-10-0-128-180.ec2.internal/pods_info.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.092975527Z nodes/ip-10-0-128-180.ec2.internal/proc_cmdline [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.093057902Z nodes/ip-10-0-128-180.ec2.internal/sysinfo.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.094750747Z nodes/ip-10-0-128-180.ec2.internal/sysinfo.tgz [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.094911567Z nodes/ip-10-0-132-31.ec2.internal/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.094937539Z nodes/ip-10-0-132-31.ec2.internal/cpu_affinities.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.095272179Z nodes/ip-10-0-132-31.ec2.internal/dmesg [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.095528595Z nodes/ip-10-0-132-31.ec2.internal/ethtool_channels [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.095601089Z nodes/ip-10-0-132-31.ec2.internal/ethtool_features [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.09577036Z nodes/ip-10-0-132-31.ec2.internal/ip-10-0-132-31.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.096337475Z nodes/ip-10-0-132-31.ec2.internal/irq_affinities.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.096485474Z nodes/ip-10-0-132-31.ec2.internal/lscpu [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.096607892Z nodes/ip-10-0-132-31.ec2.internal/lspci [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.096695817Z nodes/ip-10-0-132-31.ec2.internal/podresources.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.096807274Z nodes/ip-10-0-132-31.ec2.internal/pods_info.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.096929212Z nodes/ip-10-0-132-31.ec2.internal/proc_cmdline [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.097012187Z nodes/ip-10-0-132-31.ec2.internal/sysinfo.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.098788557Z nodes/ip-10-0-132-31.ec2.internal/sysinfo.tgz [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.098932366Z nodes/ip-10-0-142-99.ec2.internal/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.098972258Z nodes/ip-10-0-142-99.ec2.internal/cpu_affinities.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.099253386Z nodes/ip-10-0-142-99.ec2.internal/dmesg [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.099520002Z nodes/ip-10-0-142-99.ec2.internal/ethtool_channels [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.099595947Z nodes/ip-10-0-142-99.ec2.internal/ethtool_features [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.099685142Z nodes/ip-10-0-142-99.ec2.internal/ip-10-0-142-99.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.100149931Z nodes/ip-10-0-142-99.ec2.internal/irq_affinities.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.1003025Z nodes/ip-10-0-142-99.ec2.internal/lscpu [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.100424418Z nodes/ip-10-0-142-99.ec2.internal/lspci [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.100514393Z nodes/ip-10-0-142-99.ec2.internal/podresources.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.10062387Z nodes/ip-10-0-142-99.ec2.internal/pods_info.json [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.100739097Z nodes/ip-10-0-142-99.ec2.internal/proc_cmdline [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.100823012Z nodes/ip-10-0-142-99.ec2.internal/sysinfo.log [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.102573571Z nodes/ip-10-0-142-99.ec2.internal/sysinfo.tgz [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.102746951Z pod_network_connectivity_check/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.102790444Z pod_network_connectivity_check/podnetworkconnectivitychecks.yaml [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.10289074Z static-pods/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.102899701Z static-pods/kube-apiserver/ [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.11080437Z [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.110821331Z sent 71,358 bytes received 8,260,194 bytes 5,554,368.00 bytes/sec [must-gather] [must-gather-xbqkf] OUT 2026-07-16T18:58:11.110835232Z total size is 94,808,306 speedup is 11.38 [must-gather] [must-gather ] OUT 2026-07-16T18:58:11.425191625Z namespace/openshift-must-gather-kt8tj deleted [must-gather] [must-gather] [must-gather] Reprinting Cluster State: [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 5189ab8f-0cf7-4a09-8121-908b3bb10fb8 [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.24" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [git-push-artifacts] WORK_DIR: /workspace/odh-ci-artifacts [git-push-artifacts] REPO_PATH: opendatahub-io/odh-build-metadata [git-push-artifacts] REPO_BRANCH: ci-artifacts [git-push-artifacts] SPARSE_FILE_PATH: test-artifacts/docs [git-push-artifacts] SOURCE_PATH: /workspace/artifacts-dir [git-push-artifacts] DEST_PATH: test-artifacts/kserve-group-test-md6m7 [git-push-artifacts] ALWAYS_PASS: false [git-push-artifacts] configuring gh token [git-push-artifacts] taking github token from Konflux bot [git-push-artifacts] Initialized empty Git repository in /workspace/odh-ci-artifacts/.git/ [git-push-artifacts] Using partial fetch with sparse checkout for: test-artifacts/docs [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] * [new branch] ci-artifacts -> origin/ci-artifacts [git-push-artifacts] Already on 'ci-artifacts' [git-push-artifacts] branch 'ci-artifacts' set up to track 'origin/ci-artifacts'. [git-push-artifacts] TASK_NAME=kserve-group-test-md6m7-e2e-llm-inference-service [git-push-artifacts] PIPELINERUN_NAME=kserve-group-test-md6m7 [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] Already up to date. [git-push-artifacts] -rw-r--r--. 1 root 1001540000 15752167 Jul 16 18:59 /workspace/odh-ci-artifacts/test-artifacts/kserve-group-test-md6m7/e2e-llm-inference-service.tar.gz [git-push-artifacts] [ci-artifacts 9f17763] Updating CI Artifacts in e2e-llm-inference-service [git-push-artifacts] 1 file changed, 0 insertions(+), 0 deletions(-) [git-push-artifacts] create mode 100644 test-artifacts/kserve-group-test-md6m7/e2e-llm-inference-service.tar.gz [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] Already up to date. [git-push-artifacts] To https://github.com/opendatahub-io/odh-build-metadata.git [git-push-artifacts] a2f18ff..9f17763 ci-artifacts -> ci-artifacts [fail-if-needed] Failing pipeline because deploy-and-e2e step failed container step-fail-if-needed has failed : [{"key":"StartedAt","value":"2026-07-16T18:59:50.739Z","type":3}]